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EXECUTIVE SUMMARY 



The theory of sampling that is the basis for the majority of surveys conducted for the federal 
government assumes that accurate responses are obtained for all the sampled units and that the sample 
covers the target population. Surveys have always had some level of nonresponse, thus violating this 
assumption. To the extent that those who respond to surveys and those who do not are different in 
important ways, there is a potential for biases in estimates from survey data. As survey response rates 
decline, understanding the relationship between response rates and nonresponse bias has become even 
more important. Additionally, telephone survey methodologists are concerned with differences in 
telephone noncoverage rates, especially differential rates among population subgroups, such as those 
defined by region, age, race/ethnicity, and household composition, because they too can introduce bias in 
the estimates. The study described in this report was designed to examine bias in estimates from the 2007 
National Household Education Surveys Program (NHES:2007) due to nonresponse from both refusals 
and noncontact cases, as well as bias due to noncoverage of households that only had cell phones and 
households without any telephones. 

NHES is a random digit dialing (RDD) survey program developed by the National Center 
for Education Statistics (NCES) in the Institute of Education Sciences, U.S. Department of Education. It 
is designed to collect information on important educational issues through telephone surveys of 
households in the United States. NHES has enabled NCES to gather data on a wide range of issues that 
cannot be measured in traditional institution-based data collections, such as early childhood care and 
education, children’s readiness for school, parent perceptions of school safety and discipline, before- and 
after-school activities of school-age children, participation in adult and continuing education, parent 
involvement in education, school choice, homeschooling, and civic involvement. 

The sample drawn for this study to assess bias, hereinafter referred to as the Bias Study, was 
an area sample, selected independently of the NHES:2007 RDD sample. The sampled addresses for the 
Bias Study were matched to telephone numbers, and an attempt was made to gain cooperation by 
telephone when a telephone number was available. These cases followed the same telephone interviewing 
protocol as NHES:2007 RDD cases. All nonhostile’ nonresponse cases and cases that were found to be 
incorrectly matched to a telephone number or potentially incorrectly matched to a telephone number were 
sent for in-person follow-up. The field interviewer’s job was to verify that the address was a dwelling 
unit, verify the address (and confirm the telephone number if there was one) with a resident of the 
household, and attempt to gain cooperation, (i.e., complete the Screener interview). Upon gaining 



Nonhostile cases are those that were not deemed abusive or profane. 




cooperation, field interviewers would connect the respondent with the Telephone Research Center (TRC) 
on either a cellular telephone carried by the interviewer or a household telephone via a toll-free number. 
The household respondent would then complete the Screener with the TRC interviewer. The Bias Study 
data collection process for the household Screener is summarized in exhibit ES-1. Demographic 
information collected in the Screener about household members was used to determine whether anyone 
was eligible for any of the three extended interviews: the School Readiness (SR) survey of school-aged 
children, the Parent and Family Involvement in Education (PEI) survey of school-aged children, and the 
Adult Education for Work-Related reasons (AEWR) survey. The extended interviews were also 
conducted with the TRC interviewer. 



In addition to the survey data, two special data collection instruments — an Interviewer 
Observation Form (lOF) and a maximum call postcard — were designed and used for the Bias Study. 
Before approaching a sampled residence, field interviewer’s first task was to note observations about the 
neighborhood and the sampled address using the lOF. The lOF was intended to capture observations on a 
number of factors including urbanicity, neighboring area land use (e.g., residential, commercial, or 
industrial), neighborhood and household affluence, indicators of neighborhood safety or household 
security, indicators of children in the area, and language diversity. Following the field effort, when a case 
was classified as a final maximum call case, with a supervisor’s approval, field interviewers left a 
postcard that contained four questions about the household. Both the lOF and the maximum call postcard 
provided additional information about field nonrespondents that was used in the analysis. 



Among the 7,500 cases sampled for the Bias Study, a total of 5,104 were sent to the field for 
in-person attempts. Overall, 50 percent of the Screener cases finalized in the field were finalized as 
completed Screeners. Among cases with no matching telephone number and those with a mismatched 
telephone number, the percentages finalized in the field as completed Screeners were 52 percent and 54 
percent, respectively. Among the telephone maximum call, noncontact, refusal, and language problem 
cases, the percentages finalized in the field as completed Screeners were 46 percent, 45 percent, 42 
percent, and 23 percent, respectively. The final number of completed Screeners and extended interviews 
for the RDD sample and Bias Study sample are provided in table ES- 1 . 

Table ES-1. Number of completed cases by data collection stage: NHES:2007 RDD sample; 



NHES:2007 Bias Study sample 



Data collection stage 


RDD sample 


Bias Study sample 


Screener 


54,034 


4,894 


Extended interview 






School Readiness (SR) 


2,633 


292 


Parent and Family Involvement in Education (PFI) 


10,681 


1,123 


Adult Education for Work-Related Reasons (AEWR) 


7,710 


1,065 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 
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Exhibit ES-1. Data collection process for household Screeners: NHES:2007 Bias Study sample 



Bias Study 
sample / 

/ (n=7,500) / 

^ ^ ' 

▼ 

Matching 
telephone ^ 
number? 

Yes No 



T 

Attempted in 
TRC 

(n = 5,433) 



Result of initial 
telephone effort 



i 



Nonmatch 

cases 

; (n=2,067) 
L ^ t 






TRC 

respondents 

(n=2,345)* 

FT/RT 



Hostile refusals 
and final other 
nonresponse 
(n=51) / 



T 

TRC 

nonresponse 

cases 

(n=1,528) 



T 

Mismatched 

cases 

(n=1,509) / 



Attempted to gain 
cooperation in field 
and complete 
telephone interview 



' Result of^ 
field effort 



Field respondents 
(TRC nonresponse 
cases) 
(n=1,890) 



Field 

nonrespondents 
or ineligibles 
(n=2,555) 



FTO 



Field respondents 
(Nonmatch and 
Mismatch cases) 
(n=659) 

FT/RT 



Have telephone 
in household? , 



Yes 



No 



Telephone 
household field 
respondents 



Nontelephone 
household field 
respondents 



Includes 1 09 potential mismatches and 1 RDD duplicate that were not sent to the field. 

TRC: Telephone research center: FT/RT: Full treatment and reduced treatment; FTO: Full treatment only: 
n: Number of households 
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The goal of the research is to investigate bias in the estimates due to unit nonresponse and 
noncoverage, through the use of field follow-up of cases for which an interview could not be completed 
by telephone. The analysis involved two main steps. First, overall bias was evaluated by comparing final, 
adjusted estimates from the RDD sample to those from the Bias Study sample (completed extended 
interviews from FT/RT and FTO cells of figure ES-1). Since the Bias Study sample has a higher response 
rate and coverage of nontelephone households, differences between the two estimates are believed to be 
an indication of nonresponse and/or noncoverage bias in the RDD estimates. Prior to this analysis. Bias 
Study estimates were compared to external estimates from the Current Population Estimate as a check of 
their reasonableness. In addition, RDD estimates were compared before and after the weighting 
adjustments to evaluate the effect of the weighting. 

The second main step of the analysis was to evaluate the nonresponse and noncoverage bias 
components separately. Following up with nonrespondents from the telephone effort allows an 
investigation of nonresponse bias, and following up with households with no landline telephone allows an 
investigation of noncoverage bias. Specifically, nonresponse bias was evaluated by comparing estimates 
from the reduced treatment Bias Study sample (completed extended interviews from FT/RT cells of figure 
ES-1) to the frill treatment Bias Study sample (completed extended interviews from FT/RT and FTO cells 
of figure ES-1). In the reduced treatment, the TRC status is treated as the final household status for Bias 
Study cases attempted in the TRC. In the frill treatment, the final household status is based on efforts in 
both the TRC and in the field. This nonresponse study is limited because data on the households that did 
not respond to the Bias Study are not available. Thus, statements about the likely bias associated with the 
nonresponse of these households cannot be evaluated directly, and inferences about this bias are based on 
model assumptions. General characteristics of the field nonrespondents were described using the lOF and 
maximum postcard data. 

The other component of interest, noncoverage bias, was evaluated by comparing fully 
weighted estimates from the full Bias Study sample (completed extended interviews from FT/RT and 
FTO cells of figure ES-1) to estimates from the Bias Study sample restricted to telephone households 
(telephone household field respondents cell of figure ES-1). For this section, nontelephone households 
are defined as those without a landline so that the noncoverage bias associated with standard RDD 
surveys that do not include cell phones can be estimated. The NHES:2007 Bias Study survey instruments 
included a series of questions to capture the presence and number of telephone numbers in the household, 
so nontelephone households could be identified through the responses to these questions. 
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This analysis extends beyond the most common approach to bias analyses suggested in 
Seastrom (2002) in that it includes a full-scale data collection effort directed entirely at estimating bias. 
The most common methods use existing data to evaluate bias. The more typical nonresponse and 
noncoverage bias approaches have been used in the evaluation of bias in earlier NHES surveys. As a 
result, the current analysis complements and includes methods used in earlier investigations, such as 
comparisons to estimates from other surveys with higher responses. Those studies found little evidence of 
nonresponse or noncoverage bias in estimates from previous NHES surveys. 

Results from this study suggest that there is no systematic pattern of bias in key statistics 
from the NHES:2007. The comparison of the fully weighted RDD estimates to Bias Study estimates 
indicated the RDD survey underestimated the percentages of preschoolers: 

■ Who count to 20 or higher; 

■ Whose speech is often understandable to a stranger; and 

■ Who watch 2 or more hours of TV in a typical weekday; 



and overestimated: 



■ The percentage of preschoolers whose mother is not in the labor force; and 

■ The percentage of adults who are currently married. 



However, the majority of estimates evaluated showed no evidence of bias of substantive 

importance.^ 



In addition to the evaluation of overall bias in the NHES:2007 estimates, the Bias Study also 
allowed for the estimation of the nonresponse and noncoverage bias components. The NHES:2007 
estimates were produced using weights that were adjusted for nonresponse and calibrated to population 
totals — adjustments that are expected to reduce nonresponse and noncoverage bias. The results from the 
NHES:2007 Bias Study, in concert with the previous bias analyses, suggest that: 

■ Despite the falling response rates, there is no bias of substantive importance in the 
NHES:2007 estimates due to nonresponse. 



^ Bias of substantive importance is defined as a statistically significant difference of 5 percentage points or more or relative differences of 3 or 
more (i.e., when one estimate is 3 or more times larger than the other). The Bias Study was designed to allow detection of a 5 percentage point 
difference in key statistics. For NHES, this is considered a meaningful threshold to use to identity which statistically significant differences are of 
substantive importance. 
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■ A comparison of estimates before and after the raking adjustments indicated potential 
noncoverage biases in some unadjusted SR survey outcome estimates, as well as in 
some demographic estimates, that were reduced through the weighting process (see 
section 7.3 for a complete list). 

■ While the weighting adjustments appear to have reduced noncoverage bias, the Bias 
Study analysis did provide evidence of the potential for noncoverage in the RDD 
survey to result in an overestimate of the percentage of preschoolers whose parents’ 
highest educational attainment is beyond a high school diploma. 

Although estimates of noncoverage bias in other final estimates examined in this study are not of 
substantive importance as defined for this report, noncoverage bias may become more of an issue in the 
future as more households drop their landline telephone service. 
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1. INTRODUCTION 



The theory of sampling that is the basis for the majority of surveys conducted for the federal 
government assumes that accurate responses are obtained for all the sampled units and that the sample 
covers the target population. The study described in this report was designed to examine bias in the 2007 
surveys of the National Household Education Surveys Program (NHES), a random digit dial (RDD) 
telephone survey sponsored by the National Center for Education Statistics (NCES). Specifically, the 
types of bias examined are bias due to nonresponse from both refusals and noncontact cases as well as 
noncoverage bias due to the exclusion of households that only had cell phones and households without 
telephones. Surveys have always had some level of nonresponse, thus violating the assumption that 
responses are obtained for all sampled units, and the level of nonresponse has been increasing over time. 
For example, Atrostic et al. (2001) report that the rates of nonresponse were increasing for in-person 
household surveys conducted by the U.S. Census Bureau at the end of the 20th century. The response 
rates for the Survey of Consumer Attitudes, an RDD survey like the NHES, declined by an annual rate of 
three-quarters of a percentage point from 1979 to 1996, and by 1.5 percentage points per year on average 
from 1996 to 2003 (Curtin, Presser, and Singer 2005). 

To the extent that those who respond to surveys and those who do not are different in 
important ways, there is a potential for biases in estimates from survey data. As survey response rates 
decline, understanding the relationship between response rates and nonresponse bias has become 
increasingly important. One approach to understanding the relationship is to conduct nonresponse bias 
studies. The literature is replete with examples of such studies (see, for example, Boyle et al. (2002), 
Cohen and Duffy (2002), Garretsen et al. (2002), and Keeter et al. (2000)). Several years ago, NCES 
introduced a formal requirement to conduct nonresponse bias studies for its surveys when response rates 
fall below established levels for different types of data collections (Seastrom 2002), and the Office of 
Management and Budget has more recently issued guidelines that have similar requirements for all 
federal government surveys (Office of Management and Budget 2006). Although there is the potential for 
nonresponse bias in survey estimates, statistical adjustments that account for differences in response 
propensities can reduce nonresponse bias. 

Differences in telephone noncoverage rates, especially differential rates among population 
subgroups, such as those defined by region, age, race/ethnicity, and household composition, are of 
concern to telephone survey methodologists because they too can introduce bias in the estimates. The list- 
assisted RDD method used to sample telephone numbers in the NHES:2007 (Hagedom et al. (2008)) was 
shown to cover 97 percent of residential landline telephone numbers (Tucker, Eepkowski, and Piekarski 
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(2002)). The largest component of noncoverage bias in a telephone survey such as the NHES, whose 
sampling frame only includes landline phones, is probably due to the prevalence of households without 
landline telephones' and the differences between such households and those with landline telephones. 
Based on recent findings (Blumberg and Luke 2008), in the last 6 months of 2007, the percentage of 
households with no telephone service was about 2 percent, and the percentage of households with cell 
phone service alone was about 16 percent. Tucker et al. (2004) and Blumberg and Luke (2008) examined 
differences in characteristics among persons and households having no telephone service, cellular service 
only, and landline service (including both landline only, and landline and cellular). Although there are 
differences in landline noncoverage (e.g., young adults, adults in one-person households, renters, and 
Blacks and Hispanics are less likely to have landlines), statistical adjustments that account for these 
subgroup differences can reduce noncoverage bias. 

This report documents an extensive Bias Study conducted in conjunction with the 2007 
National Household Education Surveys Program (NHES:2007). The Bias Study was motivated by 
continuing declines in response rates and landline telephone coverage experienced by NHES and other 
RDD studies over time. Conducted for NCES, NHES:2007 was an RDD survey covering the 50 states 
and the District of Columbia. As discussed in chapter 3, the Bias Study sample was an area sample, 
selected independently of the NHES:2007 RDD sample. The sampled addresses for the Bias Study were 
matched to telephone numbers, and an attempt was made to gain cooperation (i.e., to complete an 
interview) by telephone when a telephone number was available. That is, households with a matched 
telephone number were first called by telephone interviewers in the Telephone Research Center (TRC). 
Those cases for which a Screener interview could not be completed by the end of the calling period 
through TRC-initiated calls were sent to the field for in-person follow-up. Households for which a 
telephone number match was not found went to the field for in-person follow-up without being called by 
TRC interviewers. As with the RDD sample, telephone data collection for the Bias Study sample began 
on January 2, 2007 and ended on May 6, 2007. In-person data collection began on March 19, 2007 and 
ended on June 24, 2007. 

The purpose of the Bias Study is to investigate bias in the estimates due to unit nonresponse 
and noncoverage, through the use of field follow-up of cases for which an interview could not be 
completed by telephone. Following up with nonrespondents from the telephone effort allows an 
investigation of nonresponse bias, and following up with households with no landline telephone allows an 
investigation of noncoverage bias. This nonresponse study is limited because data on the households that 
did not respond to the Bias Study are not available. Thus, statements about the likely bias associated with 



’ Households without landline telephones include cellular phone-only households, in addition to households with no telephone service. 
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the nonresponse of these households cannot be evaluated directly, and inferences are based on model 
assumptions. 



The analysis presented in this report differs from the standard approach to bias analyses 
suggested in Seastrom (2002) in that it involves a full-scale data collection effort directed entirely at 
estimating bias. The most common methods use existing data to evaluate bias and can be effective at 
detecting and helping to correct for it. Such methods have been used in the evaluation of bias in earlier 
NHES surveys. (See Brick (1996); Brick et al. (1997); Brick, Burke, and West (1992); Montaquila, Brick, 
and Brock (1997); Nolin et al. (2000); Nolin et al. (2004); and Roth, Montaquila, and Chapman (2007)). 
As a result, the current analysis complements earlier investigations, such as comparisons to estimates 
from other surveys with higher responses (e.g., chapter 8 in Nolin et al. (2004)). Those studies found little 
evidence of nonresponse bias or noncoverage bias in estimates from previous NHES surveys. ^ 

The remainder of this chapter provides an overview of NHES and the procedures used to 
increase response rates and minimize the effects of nonresponse in the full RDD collections, as well as a 
description of previous feasibility studies that were conducted in preparation for the Bias Study. Chapter 2 
describes the relationship between response rates and nonresponse bias, providing a theoretical 
framework for understanding nonresponse bias. Chapter 3 outlines the sample design and selection of the 
Bias Study sample, and chapter 4 describes the data collection process. Unit and item response rates for 
the Bias Study are provided in chapter 5, with some comparison to rates from the NHES:2007 RDD 
sample. Chapter 6 describes the process of creating weights and calculating standard errors for the 
analysis. The results of the bias analysis are provided in chapters 7 and 8, where chapter 7 gives an 
overview of bias, and chapter 8 addresses the components of bias. A final discussion is provided in 
chapter 9. 



1.1 National Household Education Surveys Program 



NHES is an RDD survey program developed by NCES in the Institute of Education 
Sciences, U.S. Department of Education. It is designed to collect information on important educational 
issues through telephone surveys of households in the United States. NHES has enabled NCES to gather 
data on a wide range of issues that cannot be measured efficiently in traditional institution-based data 
collections, such as early childhood care and education, children’s readiness for school, parent 



^ A difference has been found between NHES and the Current Population Survey (CPS) for the estimated percentage of children in kindergarten 
through grade 12 whose parents’ highest level of education is graduate school. The estimate from NHES has been consistently higher than that 
from the CPS. 
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perceptions of school safety and discipline, before- and after-school activities of school-age children, 
participation in adult and continuing education, parent involvement in education, school choice, 
homeschooling, and civic involvement. NHES uses computer-assisted telephone interviews (CATI) and 
has been conducted by Westat in 1991, 1993, 1995, 1996, 1999, 2001, 2003, 2005, and 2007. 

NHES provides data on populations of special interest to NCES and education researchers. 
For surveys about children, the population of interest is defined by age or grade in school, or both, for the 
particular survey topic and research questions for a given survey administration. For surveys of adults, the 
population of interest is persons ages 16 and older who are not enrolled in grade 12 or below, excluding 
those on active duty military service and those who are institutionalized. NHES targets these populations 
using specific screening and sampling procedures. 

NHES provides national cross-sectional estimates based on the 50 states and the District of 
Columbia. The NHES design also yields estimates for subgroups of interest for each survey, as defined by 
age or grade for children, educational participation status for adults, and Black and Hispanic origin for all 
populations of interest. In addition to providing cross-sectional estimates, NHES is also designed to 
provide estimates of change over time in key statistics. 

NHES:2007 was conducted from January through May 2007. Households were randomly 
sampled, and a screening interview was administered to a household respondent age 18 or older. 
Demographic information collected in the Screener about household members was used to determine 
whether anyone was eligible for the School Readiness (SR), Parent and Family Involvement in Education 
(PEI), or Adult Education for Work-Related Reasons (AEWR) Surveys. 

The SR Survey was administered to the parent or guardian^ in the household who was most 
knowledgeable about the care and education of the sampled child; sampled children were ages 3 through 
6, as of December 31, 2006, and were not yet in kindergarten."^ The PEI Survey was also administered to 
the parent or guardian who was most knowledgeable about the care and education of the sampled child; 
sampled children were ages 20 or younger, as of December 3 1 , 2006, and were enrolled or homeschooled 
in kindergarten through twelfth grade. The AEWR Survey was administered to sampled persons 16 years 
or older who were not enrolled in twelfth grade or below at the time of screening and were not 
institutionalized or on active duty in the U.S. Armed Forces. Although the AEWR data were not released 
to the public,^ the AEWR Survey was administered in both the RDD and Bias Study in order to evaluate 

^ The respondent for the SR and PFI Surveys was identified by the Screener respondent as the household member most knowledgeable about the 
care and education of the sampled child. For ease of discussion, the respondent is referred to as the parent/guardian. 

Some SR Survey items were administered about children enrolled in kindergarten through second grade. 

^ A decision was made not to release the AEWR data because of the low overall response rate. 
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bias; this was deemed important because NHES has included adult education surveys in various survey 
administrations since 1991. 

The largest component of nonresponse in most RDD surveys occurs when a household is 
first dialed and attempts are made to secure the household’s participation in the survey. This stage is 
called the Screener in NHES, and the Screener unit response rates across all the administrations of the 
survey are given in table 1-1. Screener unit response rates in 1991 and 1993 were greater than 80 percent, 
but then fell in 1995 and 1996 (73 percent and 70 percent, respectively) due, at least in part, to changes 
that increased the length and content of the screening interview (Brick and Collins 1997). In 1999, the 
length and content of the Screener were revised to be more consistent with the earlier surveys, and the 
response rate rose to 74 percent. The Screener response rates for 2001 and 2003 declined (68 percent and 
62 percent, respectively) despite the fact that the length and content of the Screener were similar to 1999. 
In 2003 a large incentive experiment was imbedded in the survey (Brick et al. 2005), and the results of 
this experiment led to the use of monetary incentives in the refusal conversion^ stage of the Screener in 
2005 (response rate of 64 percent). Despite the continued use of incentives in both an advance mailing 
and, where appropriate, a refusal conversion mailing, the Screener response rate declined to 53 percent in 
NHES:2007. 



^ Throughout this report, the term refusal conversion refers to the process of attempting to complete an interview with a case that has previously 
refused to participate. This often involves multiple call attempts and may result in contacting a household member other than the person who 
refused. 
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Table 1-1. Weighted unit response rates and percentage distribution of type of unit nonresponse for the NHES Screener:1991-2007 



Year of survey 


Number of completed 
Screeners 


Overall unit response 


Type of unit nonresponse (percentage distribution) 


rate (percent) 


Refusals 


Maximum calls 


Other nonresponse 


1991 


60,322 


81.0 


84 


7 


9 


1993 


63,844 


82.1 


68 


15 


18' 


1995 


45,465 


73.3 


84 


9 


7 


1996 


55,838 


69.9 


83 


10 


7 


1999 


55,929 


74.1 


76 


17 


7 


2001 


48,385 


67.5 


74 


18 


8 


2003 


32,049 


61.7 


76 


16 


8 


2005 


58,140 


64.2 


77 


15 


8 


2007 


54,034 


52.5 


86 


10 


4 



* The NHES:1993 percentage of other nonresponse cases is higher than that in other surveys. The lower rate of refusals and the generally higher response rate in 
NHES: 1993 are indicative of the fact that less refielding of other nonresponse cases was needed prior to ending data collection with an acceptable Screener 
response rate. 



NOTE: To avoid any differences in rates that might be attributable to the calculation method, all unit response rates given here were calculated using the business 
office method. Therefore, response rates given here are somewhat different than the official response rates cited later in this report and in other survey 
documentation. The official rates for 2001, 2003, and 2005 use the survival method. The official rate for 2007 uses the vendor-assisted method. See chapter 4 of 
Hagedom et al. (2008) for details on the methods for computing response rates. The number of household members enumerated in each data collection differed 
according to the sample requirements of the topical surveys conducted in the specific year. Maximum call cases are those that received at least eight call attempts 
during which contact was made with a person on at least one occasion, yet the Screener was not completed. Other nonresponse includes cases with language 
problems, no-answer and answering machine calls (downweighted to reflect the appropriate proportion assumed to be residential), and other forms of 
nonresponse. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), selected years, 
1991-2007. 



Procedures that have been used to increase response rates in RDD surveys include 
the following: 



■ advance letters; 

■ incentives; 

■ one or more refusal conversion attempts; 

■ special mailings such as Federal Express prior to refusal conversion; 

■ increased numbers of call attempts; 

■ leaving messages on answering machines; 

■ special training of interviewers to avoid refusals; 

■ assigning refusal cases to specially trained refusal conversion interviewers; 
and 

■ effecting changes in the protocol for working the cases, such as increasing 
total time in the field. 

All of these methods were used in NFIES:2007. The specifics of the methods used in 
NHES: 2007 are described in section 4.1 of Flagedom et al (2008). 



1.2 Previous Feasibility Studies 

Over the past several NFIES survey administrations, an increasing array of methods 
has been used in an attempt to contact potential respondents and gain cooperation, as noted 
earlier. In order to evaluate an additional method for improving survey response, the NHES: 2005 
Field Test included a test of the feasibility of conducting a national RDD survey with 
subsampling of RDD nonrespondents to be contacted for in-person interviews (see Westat 2004 
for details). Further testing of the in-person follow-up approach was conducted during the 
NHES: 2005 main study data collection and NHES:2007 field test. 

The goals of these feasibility tests were to determine whether an in-person 
component would improve response rates enough to make it a cost-effective strategy for future 
NHES surveys and to assess the feasibility of using in-person follow-up with nonrespondents and 
households without landline telephones to study nonresponse and noncoverage bias. The results 
of the initial feasibility study in 2005 were promising in some ways (field interviewers completed 
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41 percent of telephone maximum call cases and 42 percent of telephone refusal cases that were 
determined to be residential). However, some major operational issues were identified, including 
the limited utility of such an approach when the study protocol greatly limited which cases could 
be fielded for in-person collection.^ The additional study conducted in conjunction with the 
NHES:2005 RDD collection examined operational issues and identified key factors needing 
further study. It also provided information on the refusal conversion rate for the cases that were 
not subject to the restrictions imposed in the NHES:2005 Field Test, as well as estimates that 
were not subject to the compressed schedule effects of the NHES:2005 Field Test. (The 
NHES:2005 Field Test was conducted over a 2-month period; this did not allow sufficient time 
for the full data collection protocol, e.g., a 13-day hold period prior to attempting refusal 
conversion, to be followed.) 

For the study of in-person follow-up conducted with the NHES:2005 RDD 
collection, a subset of Screener cases in two predetermined sites were sent to the field after 
having been finalized on the initial refusal on the phone, after having been finalized on the third 
refusal,* and after having been finalized as maximum call or language problems. Of the 134 first 
refusal Screener cases sent to the field, 36 cases (27 percent) were completed in the field. Of the 
206 third refusal Screener cases sent to the field, 48 cases (23 percent) were completed in the 
field. Among maximum call and language problem Screener cases, 8 of the 60 cases (13 percent) 
that were sent to the field were completed. 

These studies demonstrated that field follow-up is feasible and is a viable approach 
for completing interviews with a proportion of both telephone nonrespondents and households 
without landline telephones. In preparation for both the full-scale Bias Study and the NHES: 2007 
RDD collection, a field test of NHES:2007 was conducted in 2006. For the Bias Study, the 
primary purpose of the field test was as a final test of operational procedures and materials used 
during the field follow-up. 



^ In the 2005 Field Test, only relusals that were mild and had no more than one “knowledgeable” refusal were sent to the field for an 
in-person follow-up. A knowledgeable refusal was one in which the respondent who refused heard the entire study introduction prior 
to refusing, mentioned receiving and reading the advance letter, or otherwise acknowledged that he or she understood that the study 
was a legitimate survey and chose not to participate. 

^ In NHES:2005 and NHES:2007, a subsample of Screener cases was designated to receive an abbreviated telephone protocol. A 
random subsample of cases was designated to finalize after the first refusal, while the remaining cases were designated to receive the 
full telephone protocol (which included up to three refusals before finalization). 




2. RELATIONSHIP BETWEEN RESPONSE RATES AND BIAS 



This chapter examines some theory on the relationships between response rates and 
nonresponse bias. The same theory may also be applied to noncoverage rates, although it requires some 
re-conceptualization. The relationship between response rates and bias is more complex than the 
assumption that higher response rates will produce estimates with lower nonresponse bias. The chapter 
begins by giving some theory that aids in understanding the nature of this relationship. Two approaches 
that have been examined in dealing with nonresponse bias are presented, and the approach that is most 
pertinent for this discussion is considered in more detail. As the more pertinent approach is explicated, 
connections to theories of nonresponse are provided. The chapter concludes by reviewing estimation 
methods that might be used to reduce nonresponse bias and the conditions required for these methods to 
be effective. 



2.1 Response Theories 

Two approaches to viewing nonresponse in surveys consider response as either deterministic 
or stochastic. The deterministic view implies that the population can be partitioned so that every unit can 
be classified into respondent and nonrespondent strata, irrespective of whether the unit was sampled 
(Cochran 1977, pp. 361-362). The nonresponse bias of an estimated mean depends on the sizes of the 
strata and the differences in the characteristics in the two strata. A common way of describing the 
relationship between the response rate and nonresponse bias for the mean under this approach is 



bias(y^) = (1 - rr)(Y^ ~Y„), 



( 2 . 1 ) 



where is the estimated mean based on the respondents, rr is the proportion of the population in the 
respondent stratum, is the mean of the stratum of respondents, and is the mean of the stratum of 
nonrespondents. For other statistics and estimators, the relationship varies and expressions like that given 
in (2.1) can be established. 
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The second approach takes a stochastic perspective and assumes that response to the survey 
is a random variable and that each unit has a response propensity or probability of responding, denoted 
(j)i . This response propensity model assumes that (pi > 0 for all i. When some units have zero response 
propensities, the response propensity model does not hold and these units must be treated as if they were 
not covered or using the deterministic approach. Under the response propensity model, nonresponse can 
be treated like a second phase of sampling (i.e., treating the nonrespondents as a subsample of the full 
sample), but an important difference is that response propensities, unlike sampling probabilities, are 
unknown. 



Under the stochastic model, the bias of an estimate is related to both the distribution of the 
characteristic and the distribution of the response propensities. Using the methods given in Bethlehem 
(1988) and Kalton and Maligalig (1991), the bias of the respondent mean can be written as 

bias{y^ ) = 

_ _ ( 2 . 2 ) 

where Y^ is the value of the characteristic y for unit i, Y is the population mean of the characteristic y, 
is the response propensity for unit i, cj) is the mean of the response propensities, is the standard 
deviation of the response propensities, <jy is the standard deviation of the characteristic y, is the 
correlation between the response propensities and the values of the characteristic, and N is the number of 
units in the population. Expression (2.2) shows that the respondent mean is unbiased if there is no 
correlation between the response propensity and the characteristic being estimated. 

Both the deterministic and stochastic models imply that 100 percent response results in no 
nonresponse bias. However, some relationships between response rates and bias are more obvious using 
the stochastic model. For example, suppose it were possible to raise the response propensity for every unit 
by the same factor, from to k(j)i , where k > 0 and k(j)i < 1. Using (2.2), it is clear that this type of 
increase in response rate has no effect on the bias,^ countering the assumptions that higher response rates 
necessarily result in estimates with lower nonresponse biases. In the next section, methods of estimating 
nonresponse bias using response propensity models are discussed. 



’ Note that if response propensities for every unit are increased by the factor k, then the average response propensity (f> will also increase by a 
factor of k\ thus, the factor of k that appears in the numerator and denominator of expression (2.2) will cancel. 
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2.2 



Estimating Nonresponse Bias and Bounds on the Bias 



If the relationship between the response propensities and the statistic being estimated is 
known or can be estimated, then the nonresponse bias of the estimate can be approximated. Colombo 
(2000) and Groves, Presser, and Dipko (2004) have considered this relationship and examined some of 
the consequences for bias. To provide a clear example of this approach, only estimates of a proportion are 
considered here and the goal is to estimate the proportion of the population with a characteristic (P), say 
the proportion of preschoolers who participated in center-based arrangements in the previous year. (Note 
that the proportion of the population without the characteristic is expressed as l-T*.) 

Since a proportion is a mean of a dichotomous variable (with the value of 1 indicating the 
presence of a characteristic and 0 indicating its absence), the bias of the unadjusted estimator of a 
proportion can be written using (2.2). However, for a proportion this expression can be further simplified. 
Let the average response propensity for the units with the characteristic be and the average response 
propensity for those without the characteristic be The bias in the unadjusted estimator, denoted , is 



bias{p,) = P(1 - P)(l -X){P + {\- P)X }~^ , 



(2.3) 



where T - 



-1 



(2.3): 



The bias depends only on T , the ratio of (fyi and (j)\. The following are observations based on 



■ If the average response propensities are the same for those with and without the 
characteristic (A= 1), then the estimate is unbiased irrespective of the response rate. 

■ The bias is negative when T > 1 (those with the characteristic have lower response 
propensities). 

■ The bias is positive when A < \ (those with the characteristic have higher response 
propensities). 



If A = 1, then bias(pP) = p{\ - p){\ — /(){p + (1 — p)^) ' = p(l ^ p) ' 0 ■ (p + (I - p)) * = 0 . 
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Table 2-1 gives the bias of an estimate and table 2-2 gives the relative bias of an estimate 
(bias divided by P) for different values of Because relative bias gives an indication of the magnitude of 
the bias relative to the estimate itself (rather than an absolute measure), relative bias can be useful for 
comparisons across different levels of P and To illustrate, consider a characteristic possessed by 25 
percent of the population {P = 25 percent), and suppose those without the characteristic are 10 percent 
more likely to respond than those with the characteristic {X- 1.1). Then 



biasip^) = 0.25(1 - 0.25)(1 -1.1)(0.25 + (1 - 0.25)1.1)"* = -0.017 , 
and the relative bias is bias{p P j P = - Q ,Q\1 j Q .25 = -0.07. 



Table 2-1 shows that the bias becomes larger as X increases and when P is not extreme 
(close to 0 percent or 100 percent). For characteristics possessed by 50 percent of the population, the bias 
may be large if the ratio of the response propensities for the two groups is not close to 1. In terms of 
relative bias, table 2-2 shows that when P is small, the nonresponse bias may be large relative to the size 
of the estimate, even for modest values of X. On the other hand, for P = 9Q percent, the value of X has to 
be large to result in a large relative bias. 

Table 2-1. Bias in the unadjusted estimate of a percentage, by different ratios of response 
propensities (/I = ^^ 2^1 * ) 



Ratio of response propensities for those without the characteristic to those with the 



Percent of population with 




characteristic (k) 






the characteristic (P) 


1.1 


1.5 


2.0 


2.5 


1 


-0.1 


-0.3 


-0.5 


-0.6 


5 


-0.4 


-1.6 


-2.4 


-2.9 


10 


-0.8 


-3.1 


-4.7 


-5.7 


25 


-1.7 


-6.8 


-10.7 


-13.2 


50 


-2.4 


-10.0 


-16.7 


-21.4 


75 


-1.8 


-8.3 


-15.0 


-20.5 


90 


-0.9 


-4.3 


-8.2 


-11.7 


95 


-0.5 


-2.3 


-4.5 


-6.6 


99 


-0.1 


-0.5 


-1.0 


-1.5 



NOTE: Response propensities for units with the characteristic = (j>i, those without the characteristic = 
SOURCE: Analytical derivation. 
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Table 2-2. Relative percentage bias (bias divided by the percent of the population with the 

characteristic, expressed as a percentage) for the unadjusted estimate of a percentage, 
by different ratios of response propensities (A = ^ 2^1 *) 



Percent of population with 


Ratio of response propensities for those without the characteristic to those with the 

characteristic (>^) 




the characteristic (P) 


1.1 


1.5 


2.0 


2.5 


1 


-9 


-33 


-50 


-60 


5 


-9 


-32 


-49 


-59 


10 


-8 


-31 


-47 


-57 


25 


-7 


-27 


-43 


-53 


50 


-5 


-20 


-33 


-43 


75 


-2 


-11 


-20 


-27 


90 


-1 


-5 


-9 


-13 


95 


0 


-2 


-5 


-7 


99 


0 


0 


-1 


-1 



NOTE: Response propensities for units with the characteristic = ^ 1 ; those without the characteristic = 
SOURCE: Analytical derivation. 



Since the main analysis in this report examines the existence of bias when an additional 
effort (in-person data collection) is attempted, the bias expressions are further developed to deal with this 
situation. Consider the ratio of the bias of the estimate for a survey conducted with a higher level of effort 
(say effort level = 2) to the bias for a survey conducted with a lower level of effort (effort level = 1). 
Using (2.3) and labeling the bias in the estimate using level of effort 1 as bias(p^i) and the bias using 
level of effort 2 as bias{p^2 ) ’ ^^e ratio of the biases is 



bias{p^ 2 ) + 

bias{p,i) ^(1-22) + ^ 



where j is the average response propensity for those with the characteristic at the level of effort = 1 , 
^ 2 is the average response propensity for those with the characteristic at the level of effort = 2; ^ 2,1 
^^2 ^re defined similarly for those without the characteristic; and /I, = ^2 1 / (Note 

that the expression is undefined if the bias{p^i) = 0.) 
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An easier way to understand (2.4) is to rewrite it as a percentage reduction in the bias of the 
estimate as a result of increasing the level of effort from level 1 to level 2. This percentage reduction of 
bias is 



,,100-100 p.5) 

biasyPi .^ ) T’(l - Tj ) + Tj 

Notice that if increasing data collection effort changes the response rates, but does not 
change the ratios of the average response propensities ( Tj = ^), then there is no reduction in the bias 
{k= 0 percent). At the other extreme, suppose the added effort results in /I 2 = 1 there is no bias in the 
estimate from the higher response rate survey, then /r= 100 percent. 

Table 2-3 gives the percentage reduction in bias for different values of the population 
percentage {P) holding =1.1 and varying the value of \ . The table shows the percentage reduction in 
bias is not very sensitive to P. In addition, the percentage reduction is greatest when the effort level 1 
survey estimate has a large value of /I, = ^2 1 ^ 1 ’ would be expected. It is worth noting that the bias 
due to nonresponse increases with higher response rates if X 2 > \ /f < 0. For example, suppose 

\-\.2 and then /r = -83 percent, and the nonresponse bias increases as a result of the 

additional effort. This type of result might be unusual in practice, but not impossible. 



Table 2-3. Percentage reduction in bias of estimate due to higher level of effort (x), holding 

constant the ratio of response propensities at the higher level of effort (> 1,2 =1.1) for 
different values of the ratio of response propensities at the lower level of effort (>1, ) 



Percent of population with 


Ratio of response propensities for those without the characteristic to those with the 
characteristic, at the lower level of effort (>^i) 


the characteristic (P) 


1.2 


1.5 


2.0 


2.5 


1 


45 


73 


82 


85 


5 


46 


73 


82 


85 


10 


46 


73 


83 


86 


25 


47 


74 


84 


87 


50 


48 


76 


86 


89 


75 


49 


78 


88 


91 


90 


50 


79 


89 


92 


95 


50 


80 


90 


93 


99 


50 


80 


90 


93 



NOTE: Response propensities for units with the characteristic = (j>i, without the characteristic = and the ratios 
are = ^2 \ f \ and 2^ — ^2 2 ^ ^ 2 * 

SOURCE: Analytical derivation. 
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At the time of deeiding on the level of effort to expend in a survey, it may be useful to get a 
sense of what is feasible by obtaining a bound on the reduetion in the bias. This can be done by assigning 
values for Tj and Aq. • Making reasonable assignments of these values depends on understanding the 
causes of nonresponse, especially causes that would result in response propensities that differ for those 
with and without the characteristic. Two general categories of causes of nonresponse are accessibility 
(ability to contact sampled units) and amenability (willingness of those contacted to respond). 
Mechanisms likely to give rise to different response propensities for those with and without the 
characteristic are those that are either a direct cause of the nonresponse or a variable highly correlated 
with a direct cause. For example, different response propensities due to inability to contact respondents 
might be expected in surveys estimating statistics such as travel or the use of technology to prevent 
unwanted telephone calls from reaching the household. Similarly, other characteristics, such as being in a 
single-person household, might also be highly correlated to differential response propensities due to 
inaccessibility because it is typically harder to make contact with households with only one person. 

With respect to amenability, the topic and sponsorship of the survey are features that might 
cause differential response propensities for those with and without the characteristic. This hypothesis is 
consistent with leverage-saliency theory (Groves, Singer, and Corning 1999). Advance letters and 
introductions that identify the topic and sponsor might result in persons being more or less amenable to 
participate based on whether they have a specific characteristic. For example, households without school- 
age children may be less likely to respond to a survey on education. Groves, Presser, and Dipko (2004) 
conducted some experiments using persons with known characteristics and found some support for this 
hypothesis, but the differences in response propensities for those with and without the characteristic were 
typically not large. 



2.3 Estimation Methods To Reduce Nonresponse Bias 

The relationships between response propensities and nonresponse bias presented earlier 
assume that the estimator is not subject to any nonresponse adjustments. In many surveys, auxiliary data 
are used in either standard nonresponse adjustment weighting or calibration weighting. The main goal of 
these adjustments to the base weights, which are the inverse of the selection probabilities, is often 
reduction in nonresponse bias or noncoverage bias, or both. This section explores the use of these 
adjustments and the effect the adjustments have on nonresponse bias under the response propensity model 
perspective. For additional information about these types of adjustments, see Brick and Kalton (1996), 
Kalton and Maligalig (1991), and Lundstrom and Samdal (1999). 
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Response propensity models are often explieitly used to develop nonresponse adjustments in 
surveys. Little (1986) suggested estimating response propensities and using them to ereate nonresponse 
weighting cells. He calls estimators that are formed this way response propensity stratification estimators. 
The general approach is described in Brick and Kalton (1996) within the context of other weighting 
adjustment methods. Recent examples of the use of propensity models in telephone surveys are Smith et 
al. (2004) and Blenk and Stasny (2001). 

Bethlehem (1988) and Kalton and Maligalig (1991) examined the statistical implications of 
different estimation procedures under the response propensity model of nonresponse. In particular, they 
examined the bias of the estimator when auxiliary variables are used in an attempt to reduce the bias of 
the estimator. Some theory of estimation in this case is provided below. 

The generalized regression estimator (GREG) is an estimator that uses auxiliary variables 
and is a valuable one to consider because calibration estimators such as the poststratified and raking 
estimators are asymptotically equivalent to the GREG (Deville and Samdal 1992). When all the sampled 
units respond, the GREG estimator of the mean, yQj ^ , is 

yOR ~ y HT (2-6) 

where the subscript HT indicates the Horvitz-Thompson, or inverse selection, weighted estimator (e.g., 
y HT Y. H with the probability of selection for unit i, y, the value of the characteristic y for 

unit i, s’ffie set of units in the sample, and N the total number of units in the population); X is a p-vector 
of population means of auxiliary variables; x^j- is the Horvitz-Thompson estimator of X ; f is the p- 
vector of weighted regression coefficients; and p is the number of auxiliary variables. When some units 
do not respond, a modified estimator is 



Tgs ~ Ti/r + (26 f*, (2.7) 

where the * indicates the estimator is based only on the values of the unit respondents^^ (e.g., 
y*HT - > where the sums are over the respondents). 

r r 



Expression (2.7) assumes that all unit respondents have valid responses to all items used in the estimator. As such, the values used in (2.7) 
include reported values for item respondents and imputed values for item nonrespondents. 
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Following Bethlehem (1988) the bias of (2.7) for a poststratified estimator as an estimator of 



the mean is 



bias(ylji) = X'P* - Y , (2.8) 

where is the vector of population regression parameters based on unit respondents. 

Expression (2.8) can be expressed in different forms for different estimators. Expression 
(2.2) is the simplification of (2.8) when there are no auxiliary variables (X = 1). The poststratified 
estimator (more appropriately this might be called the population weighting cell estimator as discussed in 
Brick and Kalton (1996)) is 












-1 



N 






C N- 






(2.9) 



where c = 1,...,C are the poststratification cells, is the number of units in the population in cell c, 
and ^ is the estimated mean in cell c based on the respondents only. The bias of the poststratified 
estimator given in (2.9) is 



bias(yl , ) * ^ Wj;' a a 

C 



( 2 . 10 ) 



where W^-N^/N. 

NHES uses a raking estimator to take advantage of a larger number of auxiliary variables 
than is feasible with poststratification because the number of respondents in the cells becomes sparse. For 
simplicity, a raking estimator with two dimensions or margins (denoted as c and d) is defined below, but 
the estimator can be easily extended to more dimensions. The raking estimator is 

y rk ~'^'^^cdyr,cd ’ 
c d 



Note that the subscript “c” denotes the particular cell, whereas the constant “C’ symbolizes the total number of poststratification cells. All c’s 
appearing in the expressions given here are “c” denoting the cell. 
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where y^. is the estimated mean in cell cd based on respondents only, and is an estimate of 
^cd - ^cd ! ^ (where is the number of population units in cell cd) formed by raking to the known 
marginal totals, and . 

The bias of the raking estimator is discussed in Kalton and Maligalig (1991). Two sufficient 
conditions for the raking estimator to be approximately unbiased in the presence of unit nonresponse are 



(*> 



Both conditions must hold at the same time in order for these conditions to be sufficient. The 
first condition is the poststratification condition (2.10), but in this case the covariance term is the 
covariance within the cell defined by the full cross-classification of all the raking dimensions, e.g., the 
combination of the level of the first dimension and the level of the second dimension that defines cell 
{c,d). For the two-dimensional case, the covariance between the response propensities within cell {c,d) 
and the variable values in the cell is equal to zero. The second condition requires that the mean response 
propensity for a cell of the full cross-classification can be written as a product of factors (main effects) for 
the dimensions. This implies the estimate will be biased if the response propensities involve an interaction 
effect as well as the main effects for each of the dimensions. 

The auxiliary variables used in the adjustments are assumed to be from an independent 
source, to have negligible sampling and measurement error, and to be available for the entire population. 
However, Lundstrdm and Sarndal (1999) show that standard nonresponse adjustment methods that only 
use data from the sample, such as weighting class nonresponse estimators, have similar bias properties to 
those that use population data. Thus, most of the expressions for bias are approximately valid when the 
same weighting procedures are used with sample data. For example, the bias of the nonresponse 
weighting class estimator is approximately equal to the bias of the poststratified estimator (Brick and 
Kalton 1996). 

To illustrate the role of auxiliary variables in weighting to control bias, the bias of the 
poststratified estimator is considered further. Without poststratification, the estimator is unbiased only if 
the response propensities are constant, the values of the characteristic y are constant, or there is no 
correlation between the two. With poststratification, expression (2.10) shows the estimator is unbiased if 
the response propensities are constant within the poststrata cells, if the y value is constant within the cells. 
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or if there is no correlation between the response propensities and the y values within cells. Thus, if the 
cells account for variability in either the values of the ^’s or they’s, then the poststratified estimator will 
be less biased than the unadjusted estimator. Even if the cells are not well chosen for either of these 
purposes, it is possible that the correlation between the y values and the within cells will be smaller 
than the overall correlation. Again, the result is a reduction in the bias of the estimates when the 
poststratification variables are reasonably related to the characteristics estimated. 

These arguments indicate that using data available either from the sampling frame or from 
independent sources can reduce the bias due to nonresponse. For the weighting adjustments to be 
effective at reducing the bias, the variables used in the adjustment should be related to either the 
probability of responding, the characteristics being estimated, or the correlation between the two. Even if 
the relationships between the auxiliary variables and the response propensities and the characteristics 
being estimated are not strong, the adjustments may reduce bias by attenuating the correlation. Thus, a 
reasonable strategy for bias reduction is to choose as large a number of auxiliary variables as possible in 
the adjustment. The implications of using a large number of variables on the variance of the estimates 
must also be considered, and some methods are more suitable for attaining both bias and variance 
reduction than others. For example, using response propensity scores is one way to include a large 
number of auxiliary variables in the adjustment. Raking is an estimation method suitable when there are 
many variables available to be used in adjustments. For further discussion of these topics, see. Brick and 
Kalton (1996), Fittle (1986) and Fundstrdm and Sarndal (1999). 
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3. SAMPLE DESIGN AND SELECTION FOR THE BIAS STUDY 



3.1 Overview 

A nationally representative three-stage sample of 7,500 households was selected for the 
NHES:2007 Bias Study. The three stages of selecting households are described in sections 3.2.1 to 3.2.3. 
Within sampled households, children and adults were then sampled for the School Readiness (SR), Parent 
and Family Involvement in Education (PEI), and Adult Education for Work-Related Reasons (AEWR) 
surveys using the within-household sampling algorithm developed for the main NHES:2007 random digit 
dialing (RDD) collection. The sample sizes for the Bias Study were set to allow for detection of a 5 
percentage point bias in key statistics from each of the surveys. Within-household sampling is discussed 
in section 3.2.4. 



3.2 Bias Study Sample Design 

3.2.1 Primary Sampling Unit (PSU) Selection 

At the first stage of household sample selection, 30 PSUs, defined as single counties or 
groups of a few contiguous counties, were randomly selected from the 50 states and the District of 
Columbia with probabilities of selection proportional to size. The measure of size used for PSU selection 
was the PSU population estimate for July 1, 2004, produced by the Census Bureau’s Population Estimates 
Program. One PSU was large enough that it was identified as a certainty PSU (i.e., it was brought into the 
sample with probability 1). The noncertainty PSUs were stratified by metropolitan status and, for 
metropolitan areas, further by census division and by educational attainment, as measured by the 
proportion of adults having a bachelor’s degree or higher. These characteristics were used for 
stratification because they are related to many of the survey characteristics of interest in NHES. In total, 
14 strata were created so that each had approximately the same aggregate measure of size, where the 
stratum size is the sum of the measure of size of PSUs within the stratum. The strata definitions are 
provided in exhibit 3-1. Within each stratum, two PSUs were randomly selected with probabilities 
proportionate to the measure of size. 



In one stratum, 3 PSUs were selected to achieve the total of 30 sampled PSUs (29 noncertainty PSUs and 1 certainty PSU). 
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Exhibit 3-1. Strata used for PSU selection for the NHES:2007 Bias Study 



Stratum 


Stratum definition 


Total stratum measure 
of size 


1 


Non-metropolitan 


19,767,100 


2 


Metropolitan; New England, East North Central, and West North Central 
census divisions; percentage with a bachelor’s degree or higher is less than 
22.800 


29,131,311 


3 


Metropolitan; New England, East North Central, and West North Central 
census divisions; percentage with a bachelor’s degree or higher is greater 
than or equal to 22.800 and less than 29.100 


21,981,207 


4 


Metropolitan; New England, East North Central, and West North Central 
census divisions; percentage with a bachelor’s degree or higher is greater 
than or equal to 29. 100 


21,835,548 


5 


Metropolitan; Middle Atlantic census division; percentage with a bachelor’s 
degree or higher is less than 24.310 


20,035,322 


6 


Metropolitan; Middle Atlantic census division; percentage with a bachelor’s 
degree or higher is greater than or equal to 24.310 


19,480,080 


7 


Metropolitan; South Atlantic, East South Central, and West South Central 
census divisions; percentage with a bachelor’s degree or higher is less than 
15.130 


19,356,038 


8 


Metropolitan; South Atlantic, East South Central, and West South Central 
census divisions; percentage with a bachelor’s degree or higher is greater 
than or equal to 15.130 and less than 20.775 


19,355,261 


9 


Metropolitan; South Atlantic, East South Central, and West South Central 
census divisions; percentage with a bachelor’s degree or higher is greater 
than or equal to 20.775 and less than 25.280 


19,726,185 


10 


Metropolitan; South Atlantic, East South Central, and West South Central 
census divisions; percentage with a bachelor’s degree or higher is greater 
than or equal to 25.280 and less than 29.500 


18,950,774 


11 


Metropolitan; South Atlantic, East South Central, and West South Central 
census divisions; percentage with a bachelor’s degree or higher is greater 
than or equal to 29.500 


19,412,564 


12 


Metropolitan; Mountain census division 


18,336,012 


13 


Metropolitan; Pacific census division; percentage with a bachelor’s degree 
or higher is less than 26.960 


18,460,250 


14 


Metropolitan; Pacific census division; percentage with a bachelor’s degree 
or higher is greater than or equal to 26.960 


18,564,462 



SOURCE: Retrieved from http://www.census.gov/population/estimates/metro general/Listl.txt . (3/3/2006); Census 
Bureau's Population Estimates Program, 2004. 
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3.2.2 



Segment Selection 



At the second stage of household sample selection, 1 0 area segments were selected within 
each sampled PSU with probabilities of selection proportional to size. The measure of size for segment 
selection was the number of occupied housing units in the segment, as reported in the 2000 decennial 
census. Segments were generally census blocks or block groups, but small block groups were combined 
with other block groups when necessary to form segments of sufficient size. A minimum segment size of 
100 occupied housing units was set in an effort to ensure the desired number of addresses per segment 
could be attained. Another goal of having a relatively large segment was to reduce the effects of 
clustering on the variance of estimates. 



3.2.3 Address Selection 

At the third stage of household sample selection, a two-phase sampling process was used to 
select addresses to obtain a final sample size of 250 addresses in each PSU. Lists of residential addresses 
were purchased from a vendor who maintains address lists based on the U.S. Postal Service delivery files. 
Since the vendor provides address lists by ZIP code, addresses were obtained for all ZIP codes within the 
sampled segments. Each address was then geocoded to identify the census block containing the address. 
Addresses located within census blocks that were included in the sample segments form the frame for 
address selection. In the first phase of address selection, a sample of 50 addresses was selected within 
each sampled segment, when possible. If fewer than 50 addresses from the vendor-provided list matched 
to the sampled segment, then all addresses in the segment were selected for the first phase sample. 

The group of addresses sampled in the first phase was sent to a second commercial vendor to 
be matched to white pages telephone directory listings to obtain telephone numbers, where possible. 
Overall, 59 percent of the first phase sample addresses had matching telephone numbers. This rate varied 
considerably by segment, ranging from 0 percent to 90 percent. In the second phase of sampling, within 
each sampled segment, addresses with telephone number matches were sampled at twice the rate of 
addresses without telephone number matches. The oversampling of addresses with telephone number 
matches was done to ensure a sufficient number of cases could be attempted by telephone. An equal 
number of addresses was selected within each segment in the PSU, for a total of 250 addresses per PSU. 



This occurred in fewer than 5 percent of segments. 

In most cases, 25 addresses were selected within each of the 10 segments in the PSU. However, in 7 PSUs, the address lists contained fewer 
than 25 addresses in a particular segment. In this situation, the sample sizes in the other segments in the PSU were increased to achieve the target 
of 250 sampled addresses per PSU. 
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3.2.4 



Within-Household Sampling and Precision Requirements 



For each sampled address, the primary goal was to administer a screening interview to a 
household respondent age 18 or older.'*’ As described below, the household could be contacted by phone 
or in person. Demographic information about household members collected in the Screener was used to 
determine whether anyone was eligible for the SR, PFI, or AEWR surveys. Up to three eligible persons 
(one per survey) could be sampled for each participating household. The within-household sampling 
algorithm used for the Bias Study was the same as that used for the RDD sample. The SR Survey was 
administered to the parent or guardian in the household who was most knowledgeable about the care 
and education of the sampled child; sampled children were ages 3 through 6, as of December 3 1 , 2006, 
and were not yet in kindergarten.'* The PFI Survey was also administered to the parent or guardian who 
was most knowledgeable about the care and education of the sampled child; sampled children were ages 
20 or younger, as of December 3 1 , 2006, and were enrolled or homeschooled in kindergarten through 
twelfth grade. The AEWR Survey was administered to sampled persons 16 years or older who were not 
currently enrolled in twelfth grade or below and were not institutionalized or on active duty in the U.S. 
Armed Forces. 



The sample sizes for the Bias Study were set to allow for detection of a 5 percentage point 
difference (or bias) in key statistics from each of the surveys. For example, if the work-related adult 
education participation rate from the NHES:2007 RDD survey was 40 percent, and the participation rate 
from the Bias Study was 45 percent, such a difference would be expected to be statistically significant, 
and the difference would be attributable to bias. Detection of a bias of 5 percentage points was set as the 
criterion because smaller differences are generally of less substantive importance in NHES. 



Based on address-telephone number match rates attained in NHES:2005, it was expected that 
about 60 percent of cases sampled for the Bias Study (or 4,530 of the 7,500 sampled addresses) would be 
attempted in the field because no matching telephone number would be identified or the case would not 
be completed through a telephone call initiated by an interviewer in the Telephone Research Center. 
Among cases attempted in the field, it was expected that Screeners would be completed with about 50 



Any household member age 18 or older was eligible to respond to the screening interview. However, if there were no household members ages 
18 or older, the male or female head of the household was asked to complete the Screener. Household members were defined as persons who 
considered that household as their residence, kept their possessions there, and had no other place to live. 

The respondent for the SR and PFI Surveys was identified by the Screener respondent as the household member most knowledgeable about the 
care and education of the sampled child. For ease of discussion, the respondent is referred to as the parent/guardian. 

Some SR Survey items were administered about children enrolled in kindergarten through second grade. 

The key statistics from the SR survey that were selected in order to determine necessary minimum sample sizes include participation in center- 
based care arrangements, recognition of all colors, ability to count higher than 10, knowing all letters, and ability to write own name. Key 
statistics for the PFI survey include parent participation in three or more activities in the child’s school, parent participation in home learning 
activities, and parent assessment of school practices. Key statistics for the AEWR survey include participation in adult education for work-related 
reasons and participation in employer-supported adult education. 
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percent based on the experiences in the NHES:2005 Field Test and the study of in-field follow-up 
conducted in conjunction with NHES:2005 RDD collection. This expectation took into account the longer 
field period of and the higher incentive used in the Bias Study. 

The initial target was a total of 1,144 completed extended interviews (344 SR interviews, 
400 PEI interviews, and 400 AEWR interviews), assuming unit response rates of 90 percent, 83 percent, 
and 80 percent for the SR, PEI, and AEWR surveys, respectively.^® However, aiming for these targets 
would have required changing the sampling algorithm used for the NHES:2007 RDD study substantially 
to restrict the number of persons sampled for the PEI and AEWR survey for the bias analysis. 
Additionally, prior to releasing cases for in-field follow-up, an option was under consideration in which 
Bias Study cases would be combined with RDD cases in the preparation of public-use data files. Thus, it 
was decided to use the same within-household sampling algorithm for the Bias Study that was used in the 
RDD survey. These within-household sampling rates were expected to yield 2,682 completed extended 
interviews (327 SR interviews, 1,108 PEI interviews, and 1,247 AEWR interviews). 

The first step in the within-household sampling process was to enumerate the members of 
the households. Following the enumeration, if the households had at least one preschooler, then exactly 
one was randomly sampled for the SR survey. If the household had at least one child ages 3 through 20 
enrolled in kindergarten through twelfth grade or homeschooled, then exactly one was randomly sampled 
for the PEI survey. For each survey, numbers pre-assigned at the household level were used to determine 
which child in the household should be sampled from among all children in the household eligible for the 
survey. 



Households were designated for adult sampling based on a random number. In households in 
which an adult was to be sampled, adult education participants had twice the probability of selection of 
nonparticipants. Table 3-1 shows all possible household compositions for sampling adults based on the 
presence of children in the household and adult education participation status as reported by the Screener 
respondent; the table also gives the respective domain probabilities of selection for adults or the 
likelihood that an adult would be selected, given the composition of his or her household. The maximum 
rate at which adults in households without children were sampled was 55 percent. That is, in 45 percent of 
households without children, no adult was sampled. 

Exhibit 3-2 summarizes key design features of the Bias Study. 



These expected extended interview unit response rates are slightly higher than the rates attained in recent NHES surveys of the same or similar 
subpopulations, because in-person interviews generally attain higher unit response rates than telephone interviews (Hox and de Leeuw 1994; 
Aquilino and Wright 1996; Leon et al. 2003). 
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Table 3-1. Overview of the sampling scheme for selecting adults based on household composition: 
NHES:2007 





Household eomposition 


Domain probability of seleetion 


Child in household 


Adult edueation 
partieipant 


Adult edueation 
nonpartieipant 


Adult edueation 
partieipant 


Adult edueation 
nonpartieipant 


No 






0 


0.2728 


No 






0.5456 


0 


No 






0.3637 


0.1819 


Yes 






0 


0.1364 


Yes 






0.2728 


0 


Yes 






0.1819 


0.0909 



SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Household Edueation 
Surveys Program (NHES), 2007. 



Exhibit 3-2. Key design features of the NH E S: 2007 Bias Study 



Charaeteristie 


Type of sample 


Address sample 


Mode of data eolleetion 


CATI (originating in Telephone Researeh 
Center) for eases finalized in Telephone 
Researeh Center; CATI (via eell phone or 
home phone) for eases attempted in the 
field 


Instrument 


Expanded Sereener (see seetion 4.2) and 
standard NHES:2007 extended interviews 


Respondent 


Standard NHES respondents 


Cases attempted in the field 


Follow-up with telephone nonrespondents 
and telephone nonmatehes 


Number of PSUs (sites) 


30 


Number of sereener eases attempted 




(total aeross sites) 

Expeeted number of Sereeners 


7,500 


eompleted by Telephone Researeh Center-initiated eall 
Expeeted number of Sereeners 


2,970 


eompleted in field 


2,265 


Expeeted number of eompleted extended interviews (total aeross 




sites) 


2,682 



SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Household Edueation 
Surveys Program (NHES), 2007. 
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4. BIAS STUDY DATA COLLECTION 



This chapter provides a detailed description of the data collection procedures for the Bias 
Study. It describes the telephone interviewing procedures for Bias Study cases, the recruitment and 
training of Bias Study interviewers, case priorities, procedures designed to increase respondent 
cooperation, special procedures for language problem and refusal cases, and refielding of nonresponse 
cases. A summary of the process can be found in exhibit 4-1 . 



4.1 Procedures for Bias Study Cases Prior to In-Person Efforts 

As described in section 3.2.3, all addresses in the first phase sample for the Bias Study were 
sent to a vendor to be matched with a telephone number. Addresses that were matched to telephone 
numbers were subsampled at about twice the rate of nonmatched addresses to arrive at the final (second 
phase) sample of addresses. In all, the sample consisted of 5,433 addresses matched to a telephone 
number and 2,067 cases that were not matched. The in-field procedures for the nonmatch cases are 
discussed in section 4.2. In order to evaluate nonresponse bias, cases that were matched with a telephone 
number were first attempted by telephone from the Telephone Research Center (TRC). These cases 
followed the same telephone interviewing protocol as NHES:2007 random digit dialing (RDD) cases. All 
nonhostile^^ nonresponse cases and cases that were found to be incorrectly matched to a telephone 
number or potentially incorrectly matched to a telephone number were sent for in-person follow-up. The 
telephone procedures are discussed in the next section. The non-telephone matched addresses were sent to 
the field for in-person efforts without first being tried in the TRC. 



4.1. 1 Initial Contact Procedures Prior to In-Person Efforts 

The TRC attempted to complete an interview with all households for which a telephone 
number had been matched to an address sampled for the Bias Study. Prior to calling any of the matched 
households, a letter was sent on U.S. Department of Education stationery explaining the purpose of the 
call with a $2 cash incentive enclosed to draw attention to the importance of the study. The advance letter 
provided information about the study, its sponsorship, and its purpose (appendix A). Then, using the same 

Nonhostile cases are those that were not deemed abusive or profane. 
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Exhibit 4-1. Data collection process for household Screeners: NHES:2007 Bias Study sample 



Bias Study 
sample 
(n=7,500) 



Matching 

telephone 

number? 



Yes 




I 

Attempted in 
TRC 

(n = 5,433) 



Result of initial 
telephone effort 



T 

Nonmatch 

cases 

(n=2,067) 

^ J 






TRC 

respondents 

(n=2,345)* 

FT/RT 



Hostile refusals 
and final other 
nonresponse 
(n=51) 




T 

' TRC 
nonresponse 
cases 
(n=1,528) 



T 

Mismatched 

cases 

(n=1,509) 



Attempted to gain 
cooperation in field 
and complete 
telephone interview 



Result of\^ 
field effort 



Field respondents 
(TRC nonresponse 
cases) 
(n=1,890) 



FTO 



Field 

nonrespondents 
or ineligibles 
(n=2,555) / 



Have telephone 
in household? 



Yes 

¥ 

Telephone 
household field 
respondents 




Field respondents 
(Nonmatch and 
Mismatch cases) 
(n=659) 

FT/RT 



Nontelephone 
household field 
respondents 



* Includes 109 potential mismatches and 1 RDD duplicate that were not sent to the field. 

TRC: Telephone research center; FT/RT: Full treatment and reduced treatment; FTO; Full treatment only; 
n: Number of households 
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telephone interviewing protocol followed for the NHES:2007 RDD sample, Bias Study cases were called 
by a TRC interviewer to identify households with eligible respondents. The interviewer was unaware 
whether the case was from the RDD sample or Bias Study sample. Once a household was reached, the 
interviewer administered a brief screening interview (referred to hereafter as the Screener) to a member of 
the household age 18 or older. The Screener responses determined if any of the household members 
were eligible for the SR, PFI, or AEWR surveys, and the within-household sampling algorithm was used 
to sample persons for the extended interview surveys (see section 3.2.4). 

Just as with the RDD sample, TRC-initiated telephone data collection for the Bias Study 
sample began on January 2, 2007 and ended on May 6, 2007. 



4.1.2 Assigning Cases to Telephone Interviewers 

For Bias Study cases that were matched to a telephone number and first attempted in the 
TRC, the same calling scheme used for RDD cases was used. That is, cases were prioritized for efficiency 
as follows: 



■ cases that had specific appointments; 

■ cases that had resulted in busy signals 1 5 minutes earlier; 

■ cases that had resulted in noncontact at a scheduled appointment time; 

■ cases that had unspecified appointment/general callback times for the time period; 

■ cases that had not been contacted on previous attempts and had not been attempted 
during the time period; and 

■ initial cases. 

Initial attempts to contact households and determine the presence of household members 
eligible for extended interviews were conducted in two groups separated by a one-week hold period: a 
group of four calls consisting of two evening calls, one daytime call, and one weekend call; and a group 
of three calls, consisting of two evening calls and a weekend call on a different day than the previous 
weekend call. If contact had not been made with either a household member or an answering machine 



In rare cases, a household may have no members ages 1 8 or older. In these cases, the Screener was administered to the male or female head of 
household. 
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after these two sets of calls, the case was sent to a vendor for 14 additional calls to be made by predictive 
dialing. If contact had not been made with a household member but an answering machine had been 
reached, the cycles of four calls and three calls were repeated. All cases that were no answer- answering 
machine cases were randomly subsampled to receive a total of 21 or 28 call attempts. 

Once a household member was contacted, up to 20 call attempts were made to complete the 
Screener with a household member, except in the case of language problem or refusal cases, described 
below. Once a household member was sampled as the subject of an extended interview, up to 20 call 
attempts were made to complete the interview with the identified parent/guardian respondent (for the SR 
and PFI surveys) or the sampled adult (for the AEWR survey). 

When a Screener was completed and household members were selected for extended 
interviews, the interviewer would first attempt to complete any interviews for which the Screener 
respondent was selected, because he or she was already on the telephone. If other household members 
were selected, the interviewer asked to speak with them after completing any applicable interviews (or 
making a callback appointment) with the Screener respondent. Telephone callback attempts were made as 
necessary to make contact with respondents to extended interviews. 

Non-English language/language problem cases. When English-only interviewers 
encountered a case in which the respondent indicated he or she did not speak English or had a hearing or 
speech impairment, they attempted to ascertain whether any adult household member spoke English or 
could communicate clearly enough to respond to the interview. If they were not successful, the case was 
coded one of three interim language problem statuses: hearing/speech problem, probable Spanish 
language, or another language. Specially trained interviewers recontacted the hearing/speech problem 
cases and attempted to complete an interview. Bilingual interviewers recontacted the Spanish language 
cases. Cases coded as non-English and non-Spanish were available to all interviewers, who recontacted 
the household in an effort to identify an English- or Spanish-speaking household member. If a Spanish- 
speaking household member was identified, the case was recoded as a Spanish language case and made 
available to bilingual interviewers. 



Predictive dialing is a process in which telephone numbers are automatically dialed and are routed to an attendant or operator when a telephone 
number is answered. The attendant identifies him or herself as an interviewer for the subcontractor and asks if the telephone number is for 
residential or business use. Calls resulting in no contact are not routed to an attendant or operator; they are automatically handled and classified as 
noncontact by a computer system. 

The variation in number of attempts was introduced initially for the purpose of survival method to estimate residency rates. However, the 
survival method was not ultimately used. 
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Refusal conversion. Whenever a refusal occurred, the interviewer recorded general 
demographic information about the refusing respondent (e.g., sex, approximate age) and the respondent’s 
reasons for refusing to participate if any had been given. Interviewers also rated the strength of the refusal 
as mild, firm, or hostile. Although the distinction between the mild and firm classifications is subjective, 
these two classes of refusals receive the same contact protocol; the distinction simply serves to inform 
future interviewers of the nature of the refusals. In the NHES:2007 Bias Study, mild or firm refusal cases 
were released after a 13-day hold for a conversion attempt. TRC supervisors reviewed all cases coded as 
hostile to determine whether that designation was merited. Any cases rated as hostile that were judged by 
the supervisor to be inappropriately coded were recoded to firm refusals and were eligible to be released 
for a conversion attempt. Truly hostile (profane or abusive) refusal cases were never released for 
conversion. In addition, households that contacted the National Center for Education Statistics (NCES) 
directly and declined to participate were excluded from refusal conversion. 



4.1.3 Bias Study Cases Sent for In-Person Efforts 

Of the 5,433 Bias Study cases attempted by the TRC, 2,235 were completed in the TRC. 
Another 42 cases were coded as hostile refusals and 9 were finalized as other nonresponse (e.g., the only 
member of the household was suffering from dementia as reported by the full-time nurse who answered 
the call) in the TRC and were not sent for in-person efforts. Also, in 115 cases, respondents gave a 
slightly different address than the sampled address or the respondent refused to provide the household’s 
physical address so the address could not be confirmed. A total of 23 of these 115 potential mismatches 
were initially sent to the field for address-telephone number verification. Eater, a decision was made not 
to send the remaining 92 cases out to the field based on time constraints and the results of the subsample 
of 23 cases sent to the field; approximately 74 percent (17 out of 23) were verified as correct address- 
telephone matches and the other 6 were incorrect matches. Based on the results of the subsample of 23 
cases sent to the field, the 1 09 potential mismatch cases (17 verified as correct in the field plus 92 not sent 
out) were ultimately classified as telephone respondents. A total of 2,378 out of 5,433 cases with 
telephone numbers were not sent for in-person efforts (2,235 completes, 42 hostile refusals, 9 other 
nonresponse, and 92 potential mismatch cases). 
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The 5,122 cases sent to the field included the remaining 3,055 cases first attempted in the 
TRC and the 2,067 cases without a telephone match. Table 4-1 presents the distribution of nonmatch, 
mismatch, nonresponse, and RDD duplicate cases (discussed in detail below) sent for in-person follow- 
up. 

Table 4-1, Distribution of Bias Study cases sent for in-person efforts: NHES:2007 Bias Study 



Cases sent for in-person data collection Number 

Total 5,122 

Nonmatch cases (address not matched to telephone number) 2,067 

Cases attempted but not completed by the TRC 3,055 

Mismatched cases 

Incorrect residential telephone number match 108 

Nonworking or nonresidential telephone number 1,394 

Potential address-telephone number mismatch 23 

Nonresponse cases 

Language problem 48 

Maximum call 477 

Noncontact 310 

Nonhostile refusal 693 

RDD duplicate cases 2 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 



Cases sent for in-person data collection included four kinds of cases: (1) address cases not 
matched to a telephone number, (2) address cases incorrectly matched to a telephone number, (3) 
nonresponse cases such as language problem cases, maximum call cases, noncontact cases, and refusal 
cases, and (4) a small number of cases that were duplicate RDD sample cases. The following describes 
each in turn: 



Nonmatch cases were those cases for which a telephone number could not be matched to a 
sampled address (2,067 cases). 
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Mismatched cases include the following: 



■ incorrect residential telephone match cases (108 cases) in which the address 
provided by a respondent during the telephone interview was completely different 
from the sampled address;^^ 

■ addresses matched to nonworking or nonresidential telephone numbers (1,394 
cases); and 

■ potential address-telephone mismatches (23 were sent for in-person follow-up) 
were cases attempted in the TRC for which the respondent-provided address was 
slightly different than the sampled address, or the respondent refused to provide the 
household’s physical address so the address could not be confirmed. If the address- 
telephone match was verified, the case was finalized as complete, i.e., as a telephone 
respondent. If the case was verified as a mismatch, the survey information in the 
CATI (computer-assisted telephone interviewing) system was cleaned out because it 
was collected with the wrong household. The field interviewer returned to the 
household to gain cooperation when the CATI was ready to accept information on the 
case that was sampled for the study. Six potential mismatch cases were verified as 
address-telephone number mismatches and of those six, two were completed when a 
field interviewer returned to the household. 



Nonresponse cases include the following: 



■ Language problem cases (48 cases) are those cases sent to the field that were coded 
in the TRC as households in which a language other than English or Spanish appeared 
to be spoken, or cases that were coded as hearing or speech problems. 

■ Maximum call cases (477 cases) are those cases that reached the maximum call limit 
in the TRC. The cases in the maximum call group may have had one language 
problem or up to two nonhostile refusals in its history but finalized as a maximum call 
case (i.e., up to 20 calls had been made without completing a Screener). 

■ Noncontact cases (310 cases) are those that resulted in no human contact and no 
answering machine or only answering machine contact but no human contact. 

■ Nonhostile refusal cases (693 cases) are those that refused three times on the 
telephone, or were in refusal status when the TRC data collection period closed. They 
were sent to the field as nonhostile refusals; refusal cases were sent additional 
mailings and an additional $2 in incentive cash. 



For these cases, the data collected in the TRC was cleaned out of CATI prior to sending the case to the field. 
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RDD duplicate cases were those cases for which telephone numbers sampled for the main 
RDD study and resulting in completed Screeners were duplicates of telephone numbers matched to 
sampled addresses for the Bias Study. These cases were sent to the field to confirm that the telephone 
number was matched to the correct address. One of the 2 cases was confirmed to have been completed with 
the correct address; the other was found to have been completed with a household whose address did not 
match the one sampled for the Bias Study and was re-fielded. 

All Bias Study cases were sent a letter on U.S. Department of Education stationery 
explaining the purpose of the study and its sponsor prior to in-person data collection efforts. Cases that 
were first called by a TRC interviewer before being sent to the field were sent the letter and a $2 incentive 
(previously described in section 4.1.1) when telephone data collection began. Cases that were not 
matched to a telephone number were sent a similar letter on March 13, just prior to the beginning of in- 
person field efforts. Letters sent to non-matched cases did not contain a cash incentive because field 
interviewers would be arriving at those addresses with a $20 incentive to complete the Screener (the 
standard incentive offered by the in-person interviewers, as discussed in section 4.2). 

4.1.4 Recruitment and Training of Bias Study Field Interviewers 

Recruitment of supervisors for the NHES:2007 Bias Study began in November 2006. Five 
supervisors, previously employed by Westat as field supervisors in different regions of the U.S., were 
recruited to hire and supervise field interviewers in their respectively assigned PSUs. Regions were 
roughly assigned to supervisors by each supervisor’s home location. In January 2007, supervisors began 
recruiting field interviewers for their assigned regions. Supervisors were instructed to hire two 
interviewers per PSU who were experienced in field interviewing methods. All field interviewers were 
recruited from lists of previously employed field interviewers or through connections to the supervisor. 

In-person training was held in Rockville, Maryland, on March 15, 2007, for supervisors and 
on March 16 and 17, 2007 for field interviewers. Five interviewers were trained later at various times 
over the course of the Bias Study data collection period because they were unable to attend training on the 
training dates or were hired after the field training. In total, 66 interviewers were trained, 10 of whom 
were bilingual in English and Spanish and 12 of whom were travelers who were willing to conduct 
interviews in other regions. As with TRC interviewers, supervisors and field interviewers were required to 
sign a notarized Affidavit of Nondisclosure and a Westat confidentiality pledge to adhere to the 
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confidentiality procedures outlined in the Field Worker Guide (exhibits 4-2 and 4-3), as well as undergo a 
U.S. Department of Education security background check. 

Supervisor training. Prior to training, several conference calls with all of the supervisors 
were scheduled in which field interviewer recruitment progress and in-person study procedures were 
discussed with the project director and field manager. In addition, before the in-person training mentioned 
above, supervisors were mailed a supervisors’ manual, a field interviewers’ manual, and supplemental 
printed materials to help them understand NHES, the purpose of the Bias Study, and their role as 
supervisors. Supervisors’ training consisted of a half day of informal lecture and a half day of training on 
the Basic Field Operating System (BFOS) used for managing cases. 

Field interviewer training. Field interviewers were mailed a field interviewer manual, 
along with other printed home-study materials and exercises to be completed prior to training. Twelve 
hours of classroom training for field interviewers consisted of lectures, cell-phone training, and hands-on 
role-plays and practices. At the conclusion of training all field interviewers had to successfully complete a 
certification exercise before they could begin field work. 
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Exhibit 4-2. NCES Affidavit of Nondisclosure: NHES:2007 Bias Study 



(Job Title) 



(Date of Assignment to NCES Project) 



(Organizations, State or local (NCES Data Base or File Containing 

agency or instrumentality) Individually Identifiable Information) 



(Address) 

I, , do solemnly swear (or affirm) that when given access to the 

subject NCES data base or file, I will not 

(i) use or reveal any individually identifiable information furnished, acquired, 
retrieved or assembled by me or others, under the provisions of Section 406 of 
the General Education Provisions Act (20 U.S.C. 1221e-l) for any purpose 
other than statistical purposes specified in the NCES survey, project or 
contract; 

(ii) make any disclosure or publication whereby a sample unit or survey 
respondent could be identified or the date furnished by or related to any 
particular person under this section can be identified; or 

(iii) permit anyone other than the individuals authorized by the Commissioner of 
the National Center for Education Statistics to examine the individual reports. 



(Signature) 

(The penalty for unlawful disclosure is a fine of not more than $250,000 (under 18 U.S.C. 3559 and 3571) 
or imprisonment for not more than 5 years, or both. The word “swear” should be stricken out wherever it 
appears when a person elects to affirm the affidavit rather than to swear to it.) 

State of Maryland 

County of 

Sworn and subscribed to me before a Notary Public in and for the aforementioned County and State this 
day of (year). 



(Notary Public) 
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Exhibit 4-3. Westat Confidentiality Pledge: NHES:2007 Bias Study 



WESTAT 

EMPLOYEE OR CONTRACTOR’S ASSURANCE OF CONFIDENTIALITY OF SURVEY DATA 
Statement of Policy 

Westat is firmly committed to the principle that the confidentiality of individual data obtained through Westat surveys must be protected. 

This principle holds whether or not any specific guarantee of confidentiality was given at time of interview (or self-response), or whether or not there 

are specific contractual obligations to the client. When guarantees have been given or contractual obligations regarding confidentiality have been 

entered into, they may impose additional requirements which are to be adhered to strictly. 

Procedures for Maintaining Confidentiality 

1. All Westat employees and field workers shall sign this assurance of confidentiality. This assurance may be superseded by another assurance 
for a particular project. 

2. Field workers shall keep completely confidential the names of respondents, all information or opinions collected in the course of interviews, 
and any information about respondents learned incidentally during field work. Field workers shall exercise reasonable caution to prevent 
access by others to survey data in their possession. 

3. Unless specifically instructed otherwise for a particular project, an employee or field worker, upon encountering a respondent or information 
pertaining to a respondent that s/he knows personally, shall immediately terminate the activity and contact her/his supervisor for instructions. 

4. Survey data containing personal identifiers in Westat offices shall be kept in a locked container or a locked room when not being used each 
working day in routine survey activities. Reasonable caution shall be exercised in limiting access to survey data to only those persons who 
are working on the specific project and who have been instructed in the applicable confidentiality requirements for that project. 

Where survey data have been determined to be particularly sensitive by the Corporate Officer in charge of the project or the President of 
Westat, such survey data shall be kept in locked containers or in a locked room except when actually being used and attended by a staff 
member who has signed this pledge. 

5. Ordinarily, serial numbers shall be assigned to respondents prior to creating a machine-processible record and identifiers such as name, 
address, and Social Security number shall not, ordinarily, be a part of the machine record. When identifiers are part of the machine data 
record, Westaf s Manager of Data Processing shall be responsible for determining adequate confidentiality measures in consultation with the 
project director. When a separate file is set up containing identifiers or linkage information which could be used to identify data records, this 
separate file shall be kept locked up when not actually being used each day in routine survey activities. 

6. When records with identifiers are to be transmitted to another party, such as for keypunching or key taping, the other party shall be informed 
of these procedures and shall sign an Assurance of Confidentiality form. 

7. Each project director shall be responsible for ensuring that all personnel and contractors involved in handling survey data on a project are 
instructed in these procedures throughout the period of survey performance. When there are specific contractual obligations to the client 
regarding confidentiality, the project director shall develop additional procedures to comply with these obligations and shall instruct field 
staff, clerical staff, consultants, and any other persons who work on the project in these additional procedures. At the end of the period of 
survey performance, the project director shall arrange for proper storage or disposition of survey data including any particular contractual 
requirements for storage or disposition. When required to turn over survey data to our clients, we must provide proper safeguards to ensure 
confidentiality up to the time of delivery. 

8. Project directors shall ensure that survey practices adhere to the provisions of the U.S. Privacy Act of 1974 with regard to surveys of 
individuals for the Federal Government. Project directors must ensure that procedures are established in each survey to inform each 
respondent of the authority for the survey, the purpose and use of the survey, the voluntary nature of the survey (where applicable) and the 
effects on the respondents, if any, of not responding. 

PLEDGE 



I hereby certify that I have carefully read and will cooperate fully with the above procedures. I will keep completely confidential all 
information arising from surveys concerning individual respondents to which I gain access. I will not discuss, disclose, disseminate, or provide access 
to survey data and identifiers except as authorized by Westat. In addition, I will comply with any additional procedures established by Westat for a 
particular contract. I will devote my best efforts to ensure that there is compliance with the required procedures by personnel whom I supervise. I 
understand that violation of this pledge is sufficient grounds for disciplinary action, including dismissal. I also understand that violation of the privacy 
rights of individuals through such unauthorized discussion, disclosure, dissemination, or access may make me subject to criminal or civil penalties. I 
give my personal pledge that I shall abide by this assurance of confidentiality. 



Signature 
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4.2 



Bias Study In-Person Data Collection Procedures 



In-person data collection began on March 19, 2007 and ended on June 24, 2007. Cases were 
distributed to field interviewers in three releases via signature-required FedEx. The first release of cases 
(3,601 cases in total; 2,067 of which were nonmatch cases) was shipped to field interviewers such that the 
cases would arrive at their home address on March 19. A large portion of these cases were nonmatch 
cases because many of the cases that were matched to a telephone number had yet to go through the full 
telephone-interviewing protocol in the TRC. The second release of cases was shipped to field 
interviewers around April 10, 2007 and contained 567 cases. The third release of cases was shipped 
around May 9, 2007 and contained 954 cases. All cases in the second and third releases were cases that 
had been attempted in the TRC. 

In addition to some basic office supplies, interviewers received four main tools for 
conducting the in-person follow-up. These tools included the Household Folder (HHF) (appendix B), the 
Household Information Sheet (HIS), a cellular telephone, and an NHES photo ID badge (the HHF and 
HIS form are described briefly below). Before approaching a sampled residence, field interviewers’ first 
task was to note observations about the neighborhood and the sampled address using an Interviewer 
Observation Form (lOF) (appendix C). The lOF was intended to capture observations on a number of 
factors including urbanicity, neighboring area land use (e.g., residential, commercial, or industrial), 
neighborhood and household affluence, indicators of neighborhood safety or household security, 
indicators of children in the area, and language diversity. 

Once the lOF was completed, the interviewer’s job was to verify that the address was a 
dwelling unit (defined in exhibit 4-9), verify the address label on the HHF (and telephone number if there 
was one) with a resident of the household, and attempt to gain cooperation, i.e., complete an interview. In 
order to facilitate this effort, the field interviewers offered a cash incentive of $20 for completing the 
Screener interview. Upon gaining cooperation, field interviewers would connect the respondent with the 
TRC on either a Westat-provided cellular telephone or on a household telephone via a toll-free number to 
complete the survey. The TRC interviewer would proceed with an expanded Screener. The expanded 
Screener contained additional questions about the household and its members and was only administered 
to cases sent for in-person follow-up.^® If the respondent who was selected for an extended interview had 
a household telephone and it was used to call into the TRC, the extended interview could be continued on 

The additional questions in the expanded Screener were repeated in the extended interview for sampled Screener respondents. Data from the 
additional questions in the expanded Screener were collected for possible analytic use in cases in which the extended interview was not 
completed. The question wording was similar to the corresponding question in the extended interview. 
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the household telephone and the field interviewer could leave once the Screener was completed. If the 
respondent was selected for an extended interview and did not have a household telephone or did not 
want to use the household telephone, the extended interview continued on the Westat cellular telephone. 
However, not all extended interviews in the household could be completed immediately after the 
Screener. In these instances, the interviewer either made arrangements with the TRC to call the 
respondent back at a specified time, or the field interviewer returned to the household at an agreed upon 
date and time to complete the interview. 

In addition to the study telephone, HHF, HIS, and badge, field interviewers were provided 
with supplemental materials to handle a variety of nonresponse and other situations. These materials 
included the following: 

■ “Sorry I missed you” cards indicating a call attempt (i.e., a visit to the household in 
the context of the in-person follow-up) when household members were not at home 
(appendix D); 

■ appointment cards to write an appointment time, the field interviewer’s name, and the 
study cellular telephone number at which the interviewer could be reached for the 
respondent (appendix E); 

■ study brochures; 

■ copies of the advance letter and copies of community letters to be shown to 
community officials, such as local police departments and home owner’s associations, 
to establish legitimacy; 

■ a Spanish translation card for English-speaking field interviewers to gain cooperation 
with a Spanish-speaking respondent; and 

■ Spanish versions of many of the materials for Spanish-speaking respondents. 

If, upon locating a dwelling unit, the interviewer learned that there were no residents home 
or the only person home was under 18 years of age, the interviewer was instructed to either leave a “Sorry 
I Missed Y ou” card in a discreet location or with a youth, or to ask the youth for a time when an adult 
would be available and leave an appointment card with an approximate time when the interviewer would 
return. 



In-person calls by field interviewers followed a similar time slice procedure as those 
telephone calls placed when cases were being attempted in the TRC. That is, contact attempts varied by 
day and by time to maximize the possibility of finding a respondent at home, including weekday 
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evenings, weekday daytimes, and weekends. Field interviewers made up to 10 follow-up attempts at each 
address in their assignment with the caveat that they would not revisit a household that refused in person 
without first consulting their supervisor. In order to efficiently work cases, interviewers were instructed to 
plan their household visits such that they could make calls on several cases in the same or close-by 
segments on the same trip, prioritizing appointments first, followed by new cases, and then all other cases. 

At a designated time each week, the field interviewer would hold a reporting call with a field 
supervisor. Each case the field interviewer attempted that week was reviewed with the field supervisor, 
and the field supervisor would determine if the case was to be reattempted, reassigned, or given a final 
disposition code (i.e., closed out, with no further attempts). The field supervisors then entered all contact 
attempts and results into BFOS. The NHES field manager in the Rockville office monitored the BFOS 
and held a weekly individual meeting with each supervisor to discuss any problems or field interviewer 
needs. A weekly conference call that included all field supervisors, the project director, and the field 
manager was held at the end of each week during data collection and for several weeks after data 
collection ended to discuss overall progress, any changes in protocol, new cases being sent to the field, 
and any outstanding problems. 



4.2.1 Key Data Collection Tools 

Field interviewers were supplied with a number of tools to help them gain cooperation 
during in-person efforts and to collect important observational information about each case. The key tools 
included the Interviewer Observation Form (lOF) (discussed at the beginning of section 4.2), the 
Household Folder (HHF), the Household Information Sheet (HIS), and the Field Non-Interview Report 
(NIR). The HHF, HIS, and NIR are briefly discussed below. 

Household Folder. Each HHF indicated the sampled address, the telephone number (if one 
had been matched to the sampled address), and a script that the field interviewer was to follow upon 
making contact with a household member (appendix B). Interviewers were instructed to locate the 
address, indicate the time and date of the visit, type of dwelling unit (e.g., stand alone home, townhouse 
or rowhouse, apartment, etc.), and briefly discuss the outcome of the visit, any problems encountered, or 
important observations on the back of the HHF. 
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Household Information Sheet. A Household Information Sheet (HIS)/TRC Non-Interview 
Report Form (NIRF) was enclosed with each HHF if the sampled address had been matched to a 
telephone number (exhibit 4-4). This form was one or two pages in length and indicated the sampled 
address, telephone number, and a summary of contact attempts made by the TRC indicating the date, 
time, day of the week, time slice (morning, daytime, or evening), and result of the call. This form also 
contained the comments from the TRC NIRFs (exhibit 4-5). 



Exhibit 4-4. Household Information Sheet (HIS): NHES:2007 Bias Study 

ID: FOOOOOl 

National Household Education Survey: 2007 FOLLOW-UP 
Household Information Sheet 



Loaded Address and Phone: 
1234 LUCKY RD 
FREDERICK, MD 21704 
(301 ) 555 -4231 

Result: RB 



Contact History 

DATE TIME DAY RESULTS 
02/16/2007 2:14PFRID 11 
02/23/2007 8:18 PERIL 5 5 
02/25/2007 2:26 P SUN W 3 3 
02/25/2007 2:41 P SUN W 3 3 
02/25/2007 2:56 P SUN W 42 42 
03/01/2007 2:29 P THUD 2 2 
03/13/2007 2:07PTUED 2 RB 



41 




Exhibit 4-5. Telephone Center Non-Interview Report Form (NIRF): NHES:2007 Bias Study 

( 301 ) 555 - 4231 DATE OF NIRF: 03/01/07 TIME OF NIRF: 14:29:37 

REASONS REFUSED SCREENER NIRF 

FR SAYS -SORRY WE'RE NOT ANSWERING ANY Q-AN 
D IMMEDIATELY HUNG UP BEFORE I COULD SAY ANYT 
HING 

REFUSED BY A FEMALE MILD-NO HOSTILITY 

( 301 ) 555 - 4231 DATE OF NIRF: 03/13/07 TIME OF NIRF: 14:07:48 

REASONS REFUSED SCREENER NIRF 

FR SAID SOMETHING ABT SOMEONE NOT BEING THERE 
NOW THEN HUNG UPclick 

REFUSED BY A FEMALE MILD-NO HOSTILITY 



A Field Non-Interview Report Form (NIRF) was filled out for each household where an 
interview was not completed (appendix F). Information collected on the NIRF included obtained and 
observable demographics of the would-be respondent/refuser (name, sex, age, race), type of nonresponse 
(e.g., language, maximum call, refusal, other), and reason for refusal or interview breakoff (i.e., the 
respondent decides to terminate the interview before completion). 

4.2.2 Other Field Procedures 

Some of the sampled addresses were apartment buildings, but the vendor-provided address 
did not indicate an apartment number. In situations like these, if there was a telephone match and the 
respondent confirmed that address (with the change that there should be an apartment number), then that 
household (at the address corresponding to the matching telephone number) was retained. Otherwise, 
field interviewers were instructed to collect specific information about an apartment building in order to 
randomly sample one or more apartment numbers (depending on the number of addresses originally 
sampled in the building). Specifically, interviewers counted the number of cases they had at that address, 
counted the total number of apartments in the building at a sampled address, and noted how the 
apartments were numbered (e.g., whether apartments were numbered 101-1 10, 201-210, 301-310; versus 
Al-AlO, Bl-BlO, Cl-ClO). Once interviewers compiled this information, the field manager used a 
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random sampling algorithm to select apartment numbers for the study. Field interviewers returned to the 
address to gain cooperation from those sampled units. 



4.2.3 Final Household Dispositions 

Cases could be finalized with one of several different final field disposition codes. These 
dispositions were used in BFOS to record the status of the field effort (exhibit 4-6). There were some 
discrepancies between the BFOS status and the CATI status (i.e., the final status captured by the CATI 
system) of some cases. For example, if a field interviewer left the household before the extended 
interview was completed, there may have been a breakoff or language problem of which the interviewer 
was unaware. The BFOS and CATI were reconciled at the end of data collection. Results are reported in 
table 4-2 using the final disposition codes recorded after this reconciliation was complete. However, 
composite dispositions that take into account the final TRC disposition (i.e., the CATI status at the end of 
the TRC-initiated attempts) were derived for use in weighting and computing unit response rates. Also in 
table 4-2, among cases that were sent to the field for address-phone match verification, only the field 
effort results of those cases that were found to be mismatched are reported (6 cases out of 23 potential 
mismatch cases were found to be mismatched; 1 case out of 2 RDD duplicates was found to be a 
mismatch), since for the remaining cases, the TRC-initiated results were retained. 

Completed cases (Cl, C2, C3). In-person field efforts resulted in 998 completed Screeners 
where no one in the household was sampled for an extended interview (Cl); 1,106 sampled households 
had both completed Screeners and extended interviews (C2); and 447 households had a completed 
Screener but one or more extended interviews were never completed (C3). 

Refusal cases (RB). If, during an in-person visit, a household member refused to participate, 
and the refusal was not hostile and no one in the household had refused before (in-person or by 
telephone), then a letter on U.S. Department of Education stationery was sent by FedEx to the sampled 
address further explaining the importance of the study. Field interviewers were instructed to attempt to 
gain cooperation again about two weeks after the letter had been mailed. In-person refusal letters were 
sent to 426 households during the in-person data collection. Any refusal that was deemed hostile either in 
the TRC or in person was coded as a final refusal and a field interviewer did not attempt refusal 
conversion. In-person refusal cases that had also refused at least once by telephone or had one prior in- 
person refusal (i.e., a member of the household refused twice to a field interviewer) were closed as final 
refusals. There were 1,100 final Screener refusals as the result of in-person efforts. 
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Exhibit 4-6. Final Household Field Disposition Codes: NHES:2007 Bias Study 



Final disposition 


Code 


Explanation 


Completed — ^no extended interviews 
needed 


Cl 


This eode was used when the TRC eompleted the Sereener 
and no extended interviews were required. 


Completed — Sereener and one or more 
extended interviews 


C2 


This eode was used if the respondent eompleted the 
Sereener and any or all extended interviews were also 
eompleted. 


Completed Sereener — one or more 
extended interviews outstanding 


C3 


This eode was used if the respondent eompleted the 
Sereener, and one or more extended interviews remained to 
be eompleted in the household. 


Refusal^reakoff 


RB 


This eode was used if a household member refused to 
partieipate or broke off before eompleting the Sereener with 
the TRC interviewer. 


Maximum eall 


MC 


This eode was used if a field interviewer was unable to 
make a sueeessful eontact with the household after making 
10 visits, on different days and at different times. 


Language 


LP 


This eode was used when no one in the household spoke 
English or Spanish or there was a speeeh or hearing 
problem. 


No Entry 


NE 


This eode was used for a locked building or gated 
community. 


Not Found 


NF 


This code was used when an interviewer could not locate a 
valid address. 


V aeant/Demolished/ Condemned 


NV 


A case was coded as vacant if no one lived in the residence 
or the residence was demolished or condemned. 


Not a Dwelling Unit 


ND 


This code was used when an address did not fit the 
definition of a dwelling unit. Businesses were coded “ND”. 


Other 


NO 


Any other result not described above, such as a respondent 
who was too ill to participate. 



SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Household Edueation 
Surveys Program (NHES), 2007. 
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Table 4-2. Final results of in-person data collection: NHES:2007 Bias Study 











Results of in-person efforts of Bias Study 








Cases sent for in-person data 
collection 


Total 

number 


Cl 


C2 


C3 


RB 


MC 


LP 


NE 


NF NV 


ND 


NO 


Total' 


5,104 


997 


1,106 


447 


1,101 


635 


95 


56 


14 


397 


93 


163 


Nonmatch cases (address not 
matched to telephone) 


2,067 


435 


471 


178 


379 


245 


31 


33 


9 


186 


37 


63 


Mismatched cases 
Incorrect residential 
telephone number match 


108 


22 


23 


6 


20 


14 


1 


0 


0 


11 


4 


7 


Nonworking or 
nonresidential telephone 
number 


1,394 


299 


328 


126 


243 


139 


17 


6 


3 


153 


38 


42 


Potential address-telephone 
number mismatch^ 


6 


1 


0 


1 


2 


1 


0 


0 


0 


0 


0 


1 


Nonresponse cases 
Language problem 


48 


7 


4 


0 


5 


2 


27 


0 


0 


2 


0 


1 


Maximum call 


477 


64 


100 


55 


no 


94 


7 


6 


1 


14 


6 


20 


Noncontact 


310 


48 


70 


22 


57 


64 


3 


5 


0 


22 


1 


18 


Nonhostile refusal 


693 


121 


no 


59 


285 


75 


9 


6 


1 


9 


7 


11 


RDD duplicate cases^ 


1 


0 


0 


0 


0 


1 


0 


0 


0 


0 


0 


0 



' Excludes 18 cases sent to the field for which addresses were verified as being the ones sampled for the Bias Study. 
These cases were considered to be the TRC completes. 



^ Twenty-three potential address-telephone number mismatch cases were sent to the field for address-telephone 
number match verification. Six of the cases were found to be address-telephone number mismatches and were 
attempted by field inferviewers and two of those were completed in the field. Only the field result of the six 
mismatch cases are reported in the table. 

^ Two sampled addresses for these cases were matched to telephone numbers that were identical to telephone 
numbers sampled for the main RDD NHES:2007 study. They were sent to the field to verify thaf the address- 
telephone match was correct. One was found to be an incorrect address-telephone match and is reported in the table. 
NOTE: The results of the in-person efforts are coded as follows (see exhibit 4-6 for more detailed descriptions of 
these codes): Cl, C2, C3: Screener completed; RB: Refusal/breakoff; MC: Maximum call; LP: Language, speech, or 
hearing problem; NE: No entry; NF: Not found; NV: Vacant/ demolished/condemned; ND: Not a dwelling unit; NO: 
Other. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Flousehold Education 
Surveys Program (NFIES), 2007. 



45 







Maximum call cases (MC). In order to close out a case as a maximum call, field 
interviewers must have made at least 10 in-person attempts with a minimum of three attempts on 
weekends, three attempts in the evening (after 6 p.m.) on different days of the week, two attempts in the 
afternoon on different days of the week, and two attempts in the morning on different days of the week. In 
the final two weeks of the Bias Study data collection, this maximum call criterion was reduced to five 
contact attempts, to ensure that efforts were spread across all remaining cases. When a case was classified 
as a final maximum call case, with a supervisor’s approval, the field interviewer left a postcard that 
contained four questions about the household (exhibit 4-7) and a $5 cash incentive. Out of 635 maximum 
call Screener cases, postcards were returned for 222 cases, or 35 percent of them. 

Exhibit 4-7. Maximum Call Postcard: NHES:2007 Bias Study 




National Household 

Education Surveys Program 



Please answer the following questions by placing a 
check mark (V) in the appropriate box. Your answers 
are confidential and the information will be used for 
statistical purposes only. 



SORRY WE MISSED YOU! 



How many people currently live in this household? 

□ One 

□ Two or three 

□ Four or more 



We had hoped to interview you for the National 
Household Education Surveys Program, an important 
research study sponsored by the National Center for 
Education Statistics of the U.S. Department of 
Education. The study is completed and we will not 
attempt to reach you again. However, for statistical 
purposes, will you please take a minute to complete 
this postcard? We know that your time is valuable, 
and as a token of our appreciation we are enclosing 
$5. 

After responding to the questions, please use the 
attached sticker to seal it with the mailing address on 
the outside, and then mail it back to us. If you have 
any questions about the survey, please feel free to 
contact Westat, the social science research firm 
conducting the survey at 1-888-696-5672. 



How many children (under 18 years old) currently 
live in this household? 

□ None 

□ One 

□ Two or three 

□ Four or more 



Is your home: 

□ Owned (with a mortgage, or paid in 

full)? 

□ Rented? 

□ Other arrangement? 



What is the highest level of education completed by 
anyone in your household? 

□ Bachelor’s degree or higher 

□ High school diploma 

□ Less than a high school diploma 

Thank you very much for your time. 



<Print Field ID at bottom> 
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Language problem cases (LP). The NHES interviews could be conducted in English or 
Spanish, and bilingual TRC interviewers were available to conduct the interview in either language. Field 
interviewers were instructed that, if they were not bilingual and they encountered a non-English, Spanish- 
speaking household, they could attempt to gain cooperation using a translation card (exhibit 4-8) to 
introduce the purpose of their visit and connect to a bilingual TRC interviewer. If they were unable to 
gain cooperation using the translation card, interviewers then attempted to verify the address and 
telephone information and then notify their supervisor so that the case could be reassigned to a bilingual 
field interviewer. If the language spoken in the household was something other than English or Spanish, 
the case was closed out as a “Eanguage Problem.” Ninety-five Screener cases were finalized as language 
problems due to speech or hearing difficulties or because the language spoken in the household was 
something other than English or Spanish. 



Exhibit 4-8. Translation card text: NHES:2007 Bias Study 



English text: 

I work for Westat, a social science research company in Maryland. I am working on the National 
Household Education Surveys Program, which is a research study sponsored by the U.S. Department 
of Education. Are you a member of this household and at least 18 years old? I only speak English, but 
we are conducting interviews by telephone and have Spanish-speaking interviewers who can explain 
the study to you. Using my cell phone, will you allow me to call our telephone research center so that 
you can speak with someone who speaks Spanish? We're offering $20 to households that complete 
the initial screening interview. Thank you. 



Spanish text: 

Trabajo para Westat, una firma localizada en el estado de Maryland que hace investigacion en el area 
de las ciencias sociales. Trabajo para el Programa Nacional de Encuestas en Hogares sobre 
Educacion, el cual es un estudio patrocinado por el Departamento de Educacion de los Estados 
Unidos. <j,Es usted miembro de este hogar y tiene por lo menos 18 ahos de edad? Yo hablo 
solamente ingles, pero estamos realizando entrevistas por telefono y tenemos entrevistadores que 
hablan espahol; ellos le pueden explicar de que se trata el estudio. <j,Me permite usar mi celular para 
llamar a nuestro centro de investigacion para que usted pueda hablar con alguien que habla espahol? 
Estamos ofreciendo $20 a los hogares que completan la entrevista inicial. Muchas gracias. 
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No Entry (NE), Some interviewers encountered apartment buildings that were security- 
locked or communities that were gated. Most security-locked buildings contained entryways with 
intercom systems. Field interviewers were instructed to use the intercom system to explain the reason for 
the visit if they could not gain entry. For both gated communities and locked buildings, if the interviewer 
could not gain access after several tries, or if there was no intercom system, they were instructed to 
present the Community Authorization letter (appendix A) to the manager of the building or the security 
company used to monitor the community to help explain the importance of NFIES. In 56 cases, field 
interviewers were never able to gain entry to the building or the community to complete a Screener. 

Not Found (NF). In 14 cases field interviewers could not locate an address. The code of not 
found was only used after attempting to find the address using three different resources. Additional 
resources included internet search engines, local post offices, the local fire or police departments, real 
estate agencies, and citizens in the area. 

Vacant, demolished, or condemned (NV). A dwelling unit was coded as vacant, 
demolished, or condemned if upon the field interviewer’s first visit the residence was vacant, demolished, 
or condemned. If, during their initial contact with a household, the field interviewer made contact with a 
household member and verified the address, and later returned to conduct the Screener and found that the 
household was vacant, demolished, or condemned, the case was given a final code of “NO” for other 
nonresponse, and notes about the case were made on the HHF. A total of 397 sent to the field were 
finalized as vacant, demolished, or condemned. 

Not a dwelling unit (ND). Field interviewers were expected to identify whether or not an 
address was a dwelling unit (DU) and the type of DU. In most cases, field interviewers did not have 
difficulty determining whether an address was a DU. Generally, Bias Study addresses were associated 
with a detached house, an apartment, or one house in a row of houses, such as a townhome or half of a 
duplex. Structures that did not qualify as DUs were institutional group quarters, such as a halfway house 
or other institution with 10 or more unrelated residents, military barracks and BOQs (Bachelor Officer’s 
Quarters), dormitories, penal institutions (e.g., jails, prisons), hospitals, homes for the aged, nursing 
homes, and businesses. 

Exhibit 4-9 gives the definition of a DU used for purposes of this study. A total of 93 
addresses did not fit the study definition of a DU. 
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Exhibit 4-9. Dwelling unit defined: NHES:2007 Bias Study 



A dwelling unit is a house, an apartment, a mobile home or trailer, a group of rooms or a 
single room occupied as separate living quarters or, if vacant, intended for occupancy as 
separate living quarters. 

Separate living quarters are those in which the occupants live and eat separately from any 
other individuals in the building and have direct access from the outside of the building or 
through a common hall. 



Field interviewers also indicated the type of DU structure located at the address 
(exhibit 4-10). There were 3,043 field cases that were coded as stand-alone homes; 512 were coded as 
town homes or duplexes; 1,230 were coded as apartments; and 176 were coded as something other than 
stand-alone home, town home or duplex, or apartment. The remaining cases were missing a code for DU 
type. 



Exhibit 4-10. Dwelling unit structure type: NHES:2007 Bias Study 



DU STRUCTURE TYPE 

SA Stand-alone home 
TFI Town home or Duplex 
AP Apartment, Flat 
OS Other, Specify 



Other (NO). The code “NO,” or other nonresponse, was used when all other final result 
codes did not apply. For example, when all household respondents were unavailable during the entire 
field period because of vacation or because the residence was used as a seasonal home, this code would 
be used. Other examples of when this code was used are when the only adult respondent was too ill to 
participate or when the address was initially confirmed by a resident but upon an interviewer’s second 
visit the residence was vacant. One hundred and sixty-three cases were coded “other” nonresponse. 
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4.2.4 



Known Problems 



In designing the Bias Study, Westat used past field studies as a guide for anticipating 
possible problems that could occur and for developing procedures and approaches to address these 
potential problems. This approach proved successful in heading off operational problems in most 
instances. For example, Westat developed, from experience in previous field tests, a method for sampling 
apartment numbers in instances when a sampled address should, but did not, have an apartment number 
provided by the vendor (apartment sampling procedures are described in section 4.2.2). Most of the 
problems encountered during the 2007 Bias Study were related to the fact that the Bias Study was a much 
larger, national sample compared to samples used in previous tests of in-person follow-up, and the 
previous field tests required fewer field interviewers and only one supervisor. Because the Bias Study was 
conducted on a larger scale, it presented a challenge in staffing the TRC to meet the needs of the field at 
any given time. Prior to in-person data collection, procedures were developed to coordinate field 
interviewer and telephone interviewer schedules, such as field interviewers supplying a weekly projection 
of days and times they would work and real-time text messaging to inform the TRC of actively working 
field interviewers. Despite these efforts, coordinating schedules was difficult on a large scale because 
issues such as weather could affect field operations. 

Other issues that were not apparent in previous field tests were the potential difficulties of 
tracking extended interview progress in the field. As a result of the experience in the NFIES:2007 Field 
Test, additional field codes were developed before data collection for the Bias Study to keep field 
supervisors informed of outstanding extended interviews. Flowever, the BFOS for the 2007 Bias Study 
was not developed to maintain details about the specific extended interview(s) outstanding or the 
respondent(s) sampled for the interview(s). This kind of information is important if a field interviewer is 
expected to return to a household to secure cooperation from a specific respondent. While procedures 
were implemented during data collection to meet interviewer needs in order to secure cooperation in the 
Bias Study, the process of tracking this type of information would be more efficiently and seamlessly 
gathered if collected using BFOS. 

Communication to and with field interviewers was also more difficult in a large study. In 
previous field tests, when there were fewer field interviewers and only one field supervisor, 
communication of procedural changes or respondent refusals to the TRC was more direct. Relaying 
information in real-time was much easier because there were fewer supervisors and interviewers to 
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inform. The use of laptops by field interviewers and regular checking of project email would increase the 
efficiency of communication in future studies. 

Other issues arose during the Bias Study data collection that were not related to the relative 
scale of the study in relation to past field studies. One was the relative ease for falsifying maximum call 
postcards. One way to reduce the potential for falsification of maximum call postcards is to develop a 
minimum percentage of maximum call postcards selected for validation for each field interviewer. 
Although a small number of falsified maximum call postcards were detected, these were removed from 
processing and new maximum call postcards were sent to the affected addresses. Thus, information from 
these falsified maximum call postcards has no effect on the analysis given in this report. 

4.3 Final Sample Yield 

The total numbers of completed cases by data collection stage for the RDD sample and the 
Bias Study sample are provided in table 4-3. The counts for the Bias Study reflect both the initial 
telephone data collection efforts and the in-person data collection efforts. There were a total of 54,034 
Screeners completed for the RDD survey, and 4,894 Screeners completed for the Bias Study. 



Table 4-3. Number of completed cases by data collection stage: NHES:2007 RDD sample; 
NHES:2007 Bias Study sample 



Data collection stage 


RDD sample 


Bias Study sample 


Screener 


54,034 


4,894 


Extended interview 






School Readiness (SR) 


2,633 


292 


Parent and Family Involvement in Education (PFI) 


10,681 


1,123 


Adult Education for Work-Related Reasons (AEWR) 


7,710 


1,065 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 



4.4 Characterization of Bias Study Cases 



As described above, for cases in the Bias Study sample with a matching telephone number, a 
TRC interviewer first attempted to gain cooperation (i.e., to complete an interview) by phone before 
sending cases to the field. Of the 7,500 sampled cases for the Bias Study, 2,396 were finalized (i.e.. 
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received a final disposition code) in the TRC. These include the 2,378 cases with telephone numbers that 
were not sent for in-person efforts (2,235 completes, 42 hostile refusals, 9 other nonresponse, 92 potential 
mismatch cases), plus the 17 potential mismatches and 1 RDD duplicate that were confirmed in the field 
to have been completed with the correct household over the phone. The remaining 5,104 cases were 
finalized in the field. The distribution of cases into these two groups is summarized in table 4-4. 



Table 4-4. Distribution of Bias Study cases: NHES:2007 Bias Study 



Subgroup 


Sample size 


Percent of total 


Total Bias Study sample 


7,500 


100.0 


Finalized in Telephone Research Center 


2,396 


31.9 


Completes 


2,235 


29.8 


Hostile refusals 


42 


0.6 


Other nonresponse 


9 


0.1 


Potential mismatches’ 


109 


1.5 


RDD duplicate 


1 


# 


Finalized in field^ 


5,104 


68.1 


No matching telephone number 


2,067 


27.6 


Mismatched telephone number^ 


1,509 


20.1 


Telephone nonresponse cases 


1,528 


20.4 



# Rounds to zero. 

* Includes 92 potential mismatches not sent to the field and 17 sent to the field and confirmed to have been completed 
with the correct household. 

^ Finalized in field means the case received a final disposition code in the field, where the disposition codes are 
indicated in exhibit 4-6. Finalized cases include completes as well as nonresponding and ineligible cases. 

^ Includes 1,394 cases with nonworking or nonresidential telephone numbers, 108 cases with an incorrect telephone 
number match, 6 potential mismatches determined to have been completed with the wrong household over the 
phone, and 1 RDD duplicate determined to have been completed with the wrong Bias Study case. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 



In order to assess the quality of the addresses obtained for the Bias Study sample, an address 
disposition (separate from the Screener dispositions previously discussed) was coded for each case that 
was sent to the field. Table 4-5 gives distributions of address dispositions for the Bias Study cases that 
were finalized in the field. Of the cases finalized in the field, 97 percent had a correct address (or correct 
with a minor change). This percentage was approximately equal for cases previously attempted in the 
TRC (the telephone nonrespondent columns of table 4-4) and those that were not (the nonmatch or 
mismatch columns of table 4-4). Therefore, most of the addresses from the address lists appeared to be 
valid. 
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Table 4-5. Distributions of address dispositions for Bias Study cases finalized in the field: 
NHES:2007 Bias Study 



Nonmatch or Telephone 

Total finalized in field mismateh nonrespondent 



Address disposition 


Sample 

size 


Percent 


Sample 

size 


Percent 


Sample 

size 


Percent 


Total 


5,104 


100.0 


3,576 


100.0 


1,528 


100.0 


Correct address or correct 


with minor change 


4,950 


97.0 


3,470 


97.0 


1,480 


96.9 


No entry 


56 


1.1 


39 


1.1 


17 


1.1 


Address not found 


14 


0.3 


12 


0.3 


2 


0.1 


Unable to confirm address’ 


65 


1.3 


42 


1.2 


23 


1.5 


Other^ 


19 


0.4 


13 


0.4 


6 


0.4 



' Ineludes field nonresponse eases for whieh the addresses were not elearly displayed or eonfirmed from other 
sourees. 

^ Examples inelude addresses believed to be outside of the sampled eounty and gutted apartment buildings. 

NOTE: Detail may not sum to totals beeause of rounding. 

SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Household Edueation 
Surveys Program (NHES), 2007. 



An analysis of the number of attempts required to finalize a case (i.e., to assign the case a 
final disposition code) may be used to assess and understand the amount of effort expended. The number 
of attempts to finalize a Screener is shown in figures 4-1 through 4-3 for cases finalized in the TRC, 
telephone nonresponse cases, and cases with no matching telephone number or a mismatched telephone 
number, respectively. As evident in figure 4-1, Screener completes (i.e., successfully completed Screener 
cases) were finalized quickly by phone. Also, figure 4-3 shows that more than 2,700 of the 3,576 
nonmatch/ mismatch cases received one or fewer phone attempts for the Screener. This indicates that 
instances of mismatched phone numbers were often identified on the first phone attempt, where they were 
determined to be a mismatch either because the phone number was nonworking or nonresidential or the 
address provided by the respondent in the telephone interview did not match the sampled address. For 
telephone nonresponse cases (figure 4-2), a large number of attempts were made in the TRC before 
sending the cases to the field. 
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Figure 4-1. Total number of attempts to finalize a Screener for cases finalized in the TRC: 
NHES:2007 Bias Study 

Number of cases 




15 20 than 

20 

Number of attempts 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 
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Figure 4-2. Total number of attempts to finalize a Screener for telephone nonresponse cases: 
NHES:2007 Bias Study 



Number of cases 




20 

Number of attempts 



♦ Overall 

— — Phone 

Field 



NOTE: Overall number of attempts is the sum of the number of phone attempts to finalize a Sereener and the 
number of field attempts to finalize a Sereener for a given ease. 

SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Household Edueation 
Surveys Program (NHES), 2007. 
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Figure 4-3. Total number of attempts to finalize a Screener for nonmatch and mismatch cases: 
NHES:2007 Bias Study 
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NOTE: Overall number of attempts is the sum of the number of phone attempts to finalize a Screener and the 
number of field attempts to finalize a Screener for a given case. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 
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Figure 4-4. Number of attempts to ascertain residential status of household: NHES:2007 Bias 
Study 



Number of cases 




10 

Number of attempts 



♦ Nonmatch and mismatch 

— — Noncontact in TRC 

- Other 



NOTE: The “Other” eategory eonsists of eases for whieh the resideney status was determined in the initial TRC 
effort, i.e. eases that were finalized in the initial TRC effort and eases that were sent to the field after having been 
eoded a language problem, maximum eall, or refusal in the TRC. 

SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Household Edueation 
Surveys Program (NHES), 2007. 



It took an average of 3.8 attempts to finalize a Screener as complete, nonresponse, or 
nonresidential in the field. As shown in table 4-6, this varied by the final household field disposition, with 
nonrespondents requiring almost twice as many attempts as completes, and over twice as many attempts 
as cases finalizing as nonresidential. Having not been previously attempted by phone, cases with no 
matching phone number required a greater number of attempts in the field to finalize as complete or 
nonresponse than the telephone nonresponse cases (e.g., language problem, maximum call, noncontact, 
and refusal). No such pattern is evident for final nonresidential dispositions. In general, households that 
had refused the Screener in the TRC required fewer attempts overall to obtain a final disposition code 
than the other TRC results. 
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Table 4-6. Percentage of cases finalized in field and mean number of field attempts, by final household field disposition and TRC result: 
NHES:2007 Bias Study 





Total finalized 


Percentage of cases finalized in field 






Mean number of field attempts 




TRC result 


in field 


Completes 


Nonrespondents Nonresidential 


Total 


Completes 


Nonrespondents Nonresidential 


Total 


5,104 


50.0 


40.2 


9.9 


3.8 


2.9 


5.3 


2.4 


No matching telephone 


number 


2,067 


52.4 


36.3 


11.2 


4.1 


3.2 


6.0 


2.5 


Mismatched telephone 
number’ 


1,509 


53.5 


32.7 


13.9 


3.7 


2.9 


5.5 


2.3 


Language problem 


48 


22.9 


72.9 


4.2 


3.1 


3.0 


3.1 


3.0 


Maximum call 


477 


45.9 


49.7 


4.4 


3.9 


2.7 


5.2 


1.6 


Noncontact 


310 


45.2 


47.4 


7.4 


4.0 


2.6 


5.6 


3.0 


Refusal 


693 


41.7 


55.8 


2.5 


2.9 


2.2 


3.5 


1.5 



' Excludes the 17 potential mismatches and 1 RDD duplicate that were verified as having been completed with the correct household in the TRC. 

NOTE: The percentage of cases finalized in the field by final household disposition may not sum to 100 because of rounding. The final household field 
dispositions correspond to the codes in exhibit 4-6 as follows: Completes: Cl, C2, and C3; Nonrespondents: RB, MC, LP, NE, and NO; Nonresidential: NF, NV, 
and ND. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Flousehold Education Surveys Program (NFIES), 2007. 



Residency status (i.e., whether the address is residential or nonresidential) was generally 
determined on the first attempt. The number of attempts to ascertain residency status is shown in figure 
4-4. The number of attempts is plotted separately for cases with no matching telephone number or a 
mismatched telephone number, cases coded as non-contact in the TRC before being sent to the field, and 
all other Bias Study cases. For nonmatch or mismatch cases and cases coded non-contact in the TRC, 
the number of attempts reflects the field effort only, since residency status was never determined by 
phone. 



The number of refusals for cases finalized in the TRC, telephone nonresponse cases, and 
cases with no matching telephone number or a mismatched telephone number, respectively, is shown in 
table 4-7. Most of the refusals were by phone. Overall refusal for telephone nonresponse cases and 
nonmatch and mismatch cases is the sum of the number of refusals by telephone and the number of 
refusals in field before a give case was finalized in field. For example, if a case ever refused to finalize a 
Screener once by telephone and once in field, it was counted as a case with 2 refusals in table 4-7. 

Characteristics of the three sets of Bias Study cases that completed an extended interview — 
cases finalized in the TRC, telephone nonresponse cases, and nonmatches and mismatches — are provided 
in tables 4-8 to 4-10 for SR respondents, PFI respondents, and AEWR respondents, respectively. The 
frequencies are unweighted and are intended only as a description of the sample. A limited comparison of 
the three sets of cases is provided below as a first look at potentially noteworthy differences. However, 
the purpose of the frequencies is not to provide an evaluation of bias. Weighted frequencies for the 
evaluation of bias are provided in chapters 7 and 8. The percentages reflect skip patterns; the 
denominators include only respondents to the item. 



All other Bias Study cases consist of cases for which the residency status was determined in the initial TRC effort (i.e., cases that were finalized 
in the initial TRC effort and cases that were sent to the field after having been coded a language problem, maximum call, or refusal in the TRC). 
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Table 4-7. Number and percentage of cases, by number of refusals and subgroups: NHES:2007 Bias Study 



ON 

o 



Cases finalized in 
the Telephone 

Researeh Center Telephone nonresponse eases Nonmateh and mismateh eases 



Refusals by 
telephone 


Overall refusals 


Refusals by 
telephone' 


Refusals in field Overall refusals 


Refusals by 
telephone' 


Refusals in field 



Cases elassified 
by number of 
refusals 


Sample 

size 


Pereent 


Sample 

size 


Pereent 


Sample 

size 


Pereent 


Sample 

size 


Pereent 


Sample 

size 


Pereent 


Sample 

size 


Pereent 


Sample 

size 


Pereent 


Total 


2,396 


t 


1,528 


t 


1,528 


t 


1,528 


t 


3,576 


t 


3,576 


t 


3,576 


t 


Cases with 
0 refusal 


1,438 


60.0 


398 


26.0 


503 


32.9 


1,023 


67.0 


2,633 


73.6 


3,387 


94.7 


2,759 


77.2 


Cases with 
1 refusal 


633 


26.4 


211 


13.8 


175 


11.5 


444 


29.1 


593 


16.6 


134 


3.7 


553 


15.5 


Cases with 
2 or more 
refusals 


325 


13.6 


919 


60.1 


850 


55.6 


61 


4.0 


350 


9.8 


55 


1.5 


264 


7.4 



t Not applieable. 

* Primarily refusals obtained in the initial TRC data eolleetion effort, but also ineludes refusals over the telephone during the in-person effort. 
NOTE: Detail may not sum to totals beeause of rounding. 

SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Household Edueation Surveys Program (NHES), 2007. 



Table 4-8. Characteristics of Bias Study School Readiness (SR) respondents: NHES:2007 Bias 
Study 



Finalized in Telephone Telephone Nonmatehes and 

Researeh Center nonrespondents mismatehes 



Sample Sample Sample 



Charaeteristie 


size 


Estimate 


size 


Estimate 


size 


Estimate 


Total 


103 


100.0 


47 


100.0 


142 


100.0 


Partieipation in eenter-based eare 


66 


64.1 


34 


72.3 


77 


54.2 


Reeognizes all eolors 


86 


83.5 


42 


89.4 


121 


85.2 


Counts to 20 or higher 


69 


67.0 


33 


70.2 


96 


67.6 


Knows all letters 


39 


37.9 


15 


31.9 


39 


27.5 


Can write own name 


67 


65.0 


35 


74.5 


91 


64.1 


Census region 


Northeast 


31 


30.1 


15 


31.9 


16 


11.3 


Midwest 


31 


30.1 


6 


12.8 


36 


25.4 


South 


28 


27.2 


12 


25.5 


50 


35.2 


West 


13 


12.6 


14 


29.8 


40 


28.2 


Home ownership 


Own 


84 


81.6 


36 


76.6 


81 


57.0 


Rent/other 


19 


18.4 


11 


23.4 


61 


43.0 


Age 


3-4 


85 


82.5 


41 


87.2 


118 


83.1 


5-6 


18 


17.5 


6 


12.8 


24 


16.9 


Grade 


Not enrolled 


36 


35.0 


14 


29.8 


62 


43.7 


Presehool 


67 


65.0 


33 


70.2 


80 


56.3 



NOTE: Detail may not sum to totals beeause of rounding. 

SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Flousehold Edueation 
Surveys Program (NHES), 2007. 



61 





Table 4-9. Characteristics of Bias Study Parent and Family Involvement in Education (PFI) respondents: NHES:2007 Bias Study 





Finalized in Telephone 
Researeh Center 


Telephone 

nonrespondents 


Nonmatehes and 
mismatehes 


Charaeteristie 


Sample size 


Estimate 


Sample 

size Estimate 


Sample 

size 


Estimate 


Total 


498 


100.0 


164 


100.0 


461 


100.0 


Parents partieipate in 5 or more aetivities in the ehild’s sehool’ 


249 


51.1 


88 


54.3 


229 


50.4 


Parents report sehool provides information very well 


About how ehild is doing in sehool 


299 


61.4 


100 


61.7 


269 


59.3 


About how to help ehild with his/her homework 


224 


46.0 


76 


46.9 


225 


49.6 


About why ehild is plaeed in partieular groups or elasses 


210 


43.1 


77 


47.5 


209 


46.0 


About how to help ehild plan for eollege or voeational sehool 


116 


41.0 


32 


36.0 


89 


37.6 


About the family's expeeted role at ehild’s sehool 


239 


49.1 


79 


48.8 


221 


48.7 


Parents took 3 or more outings with ehild in the past month^ 


254 


51.0 


87 


53.0 


218 


47.3 


Parents eheek to see that ehild's homework gets done 


365 


79.0 


137 


90.1 


383 


90.8 


Census region 


Northeast 


no 


22.1 


50 


30.5 


86 


18.7 


Midwest 


146 


29.3 


29 


17.7 


95 


20.6 


South 


132 


26.5 


54 


32.9 


157 


34.1 


West 


no 


22.1 


31 


18.9 


123 


26.7 


Home ownership 


Own 


431 


86.5 


129 


78.7 


277 


60.1 


Rent/other 


67 


13.5 


35 


21.3 


184 


39.9 



See notes at end of table. 



Table 4-9. Characteristics of Bias Study Parent and Family Involvement in Education (PFI) respondents: NHES:2007 Bias Study — 
Continued 



Finalized in Telephone Nonmatehes and 

Telephone Researeh Center nonrespondents mismatehes 



Sample Sample 



Charaeteristie 


Sample size 


Estimate 


size 


Estimate 


size 


Estimate 


Age 


3-6 


62 


12.4 


21 


12.8 


67 


14.5 


7-8 


65 


13.1 


25 


15.2 


84 


18.2 


9-10 


69 


13.9 


27 


16.5 


61 


13.2 


11-12 


73 


14.7 


20 


12.2 


64 


13.9 


13-14 


70 


14.1 


23 


14.0 


63 


13.7 


15-16 


93 


18.7 


30 


18.3 


83 


18.0 


17-20 


66 


13.3 


18 


11.0 


39 


8.5 


Grade 


Kindergarten 


40 


8.0 


15 


9.1 


36 


7.8 


1-2 


56 


11.2 


23 


14.0 


82 


17.8 


3-4 


80 


16.1 


23 


14.0 


77 


16.7 


5-6 


72 


14.5 


23 


14.0 


59 


12.8 


7-8 


74 


14.9 


26 


15.9 


74 


16.1 


9-10 


80 


16.1 


26 


15.9 


73 


15.8 


11-12 


96 


19.3 


28 


17.1 


60 


13.0 



' Any five or more of the following: Attended a general sehool meeting (FSMTNG); attended a meeting of the parent-teaeher organization or assoeiation 
(FSPTMTNG); went to a regularly seheduled parent-teaeher eonferenee with the ehild’s teaeher (FSATCNFN); attended a sehool or elass event, sueh as a play, 
danee, sports event, or seienee fair beeause of the ehild (FSSPORT); served as a volunteer in the ehild’s elassroom or elsewhere in the sehool (FSVOL); 
partieipated in fundraising for the sehool (FSFUNDRS); served on a sehool eommittee (FSCOMMTE); and met with a guidanee eounselor in person 
(FSCOUNSLR). 

^ Any three or more of the following: Visited a library (FOLIBRAY); visited a bookstore (FOBOOKST); went to a play, eoneert, or other live show 
(FOCONCRT); visited an art gallery, museum, or historieal site (FOMUSEUM); visited a zoo or aquarium (FOZOO); attended an event sponsored by a 
eommunity, religious, or ethnie group (FOGROUP); and attended an athletie or sporting event (outside of sehool) in whieh the ehild was not a player 
(FOSPRTEV). 

NOTE: Detail may not sum to totals beeause of rounding. 

SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Flousehold Edueation Surveys Program (NHES), 2007. 



Table 4-10. Characteristics of Bias Study Adult Education for Work-Related Reasons (AEWR) respondents: NHES:2007 Bias Study 



Finalized in Telephone Telephone Nonmatehes and 

Researeh Center nonrespondents mismatehes 



Sample Sample Sample 



Charaeteristie 


size 


Estimate 


size 


Estimate 


size 


Estimate 


Total 


530 


100.0 


134 


100.0 


401 


100.0 


Partieipation in adult education for work-related reasons 


221 


41.7 


62 


46.3 


212 


52.9 


Partieipation in distance education 


130 


57.0 


34 


53.1 


121 


54.5 


Sex 


Male 


203 


38.3 


43 


32.1 


185 


46.1 


Female 


327 


61.7 


91 


67.9 


216 


53.9 


Adult education participation status from screener 


Participant 


268 


50.6 


72 


53.7 


225 


56.1 


Nonparticipant 


262 


49.4 


62 


46.3 


176 


43.9 


AEWR respondent was Screener respondent 


Yes 


169 


31.9 


34 


25.4 


104 


25.9 


No 


361 


68.1 


100 


74.6 


297 


74.1 


Census region 


Northeast 


143 


27.0 


26 


19.4 


90 


22.4 


Midwest 


132 


24.9 


28 


20.9 


75 


18.7 


South 


129 


24.3 


54 


40.3 


131 


32.7 


West 


126 


23.8 


26 


19.4 


105 


26.2 


Home ownership 


Own 


467 


88.1 


101 


75.4 


226 


56.4 


Rent/other 


63 


11.9 


33 


24.6 


175 


43.6 



NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2007. 



The three Bias Study groups are similar on certain key characteristics, such as whether the 
SR-eligible child can count to 20 or higher, whether the parents participate in 5 or more activities at the 
PFI-eligible child’s school, and the participation in distance education for adults. For example, 67 percent 
of SR respondents who finalized in the TRC indicated that their children were able to count to 20 or 
higher, compared to 70 percent for the telephone nonrespondents and 68 percent for the nonmatches and 
mismatches. 

Flowever, there are some differences between the groups. For instance, the rate of home 
ownership is lower for cases that had no matching phone number or a mismatched phone number. For PFI 
respondents, the home ownership rate was 60 percent for cases with nonmatches and mismatches, 
compared to 87 percent for cases finalized in the TRC, and 79 percent for telephone nonrespondents. 
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5. UNIT AND ITEM RESPONSE RATES 



Unit and item nonresponse are generally regarded as important measures of survey quality. 
Response rates are of concern because survey estimates could potentially suffer from nonresponse bias if 
those who respond to a survey or item are very different from those who do not. This chapter presents the 
unit and item response rates for the 2007 National Household Education Surveys Program (NHES:2007) 
Bias Study. Unit response rates are defined (section 5.1) and an analysis of unit response rates is 
presented for the Screener (section 5.2). Unit response rates are also given for each of the three extended 
interview surveys, the School Readiness Survey (SR), the Parent and Family Involvement in Education 
Survey (PEI), and the Adult Education for Work-Related Reasons Survey (AEWR) (section 5.3). Item 
response rates are discussed in section 5.4. 

5.1 Definition of Unit Response Rates 

A unit response rate is the ratio of the number of units with completed interviews (for 
example, the units could be telephone numbers, households, or persons) to the number of units sampled 
and eligible for the interview. In some cases, these rates are easily defined and computed, while in other 
cases the denominator of the ratio must be estimated. 

For reporting the results from the Bias Study, the overall unit response rate indicates the 
percentage of possible interviews that were completed taking all survey stages into account, while the unit 
response rate measures the percentage of interviews that were completed for a specific stage of the survey. 
Specifically, household members were identified for interviews in a two-stage process, with a Screener 
interview conducted before an extended interview. Screener interviews were conducted to enumerate and 
sample household members, and then questionnaires were administered for the sampled members. If the 
responding household member failed to complete the first-stage Screener, no members could be sampled for 
other interviews. Under this design, the unit response rate for the first stage is the estimated percentage of 
households that completed the Screener. The unit response rate for the second stage (SR, PEI, or AEWR 
interviews) is the percentage of sampled persons that completed these extended interviews. The overall 
unit response rate is the product of the first- and second-stage unit response rates (i.e., the Screener unit 
response rate multiplied by the extended interview unit response rate). 
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Unit response rates and overall unit response rates can be either unweighted or weighted. 
The unweighted rate, computed using the raw number of cases, provides a useful measure of the success 
of the operational aspects of the survey. That is, the unweighted rate indicates the proportion of residential 
cases attempted that were successfully completed. The weighted rate, computed by summing the base 
weights (usually the reciprocals of the probability of selecting the units) for both the numerator and 
denominator, gives a better measure of the success of the survey with respect to the population sampled 
since the weights allow for inference of the sample data (including response status) to the population 
level. When analyzing survey data, estimates are typically computed using adjusted survey weights. (See 
chapter 6 for a discussion of the computation of the adjusted survey weights for the Bias Study.) 
However, weighted unit response rates, which include sums of weights of both unit respondents and unit 
nonrespondents, are computed using base weights. The unweighted and weighted unit response rates are 
usually similar unless the probabilities of selection and the unit response rates in the categories with 
different selection probabilities vary considerably. All of the unit response rates discussed in this chapter 
are base weighted unless noted specifically in the text, since the main purpose of this chapter is to 
describe the success of the survey with respect to the survey population. 

5.2 Screener Unit Response Rates 

The unit response rates use the weighted number of responding households as the numerator, 
and the total number of households as the denominator. For the RDD sample, the number of households 
needs to be estimated because the residential status of all the telephone numbers is not known. Various 
approaches are available; the vendor-assisted method was used for the NHES:2007 RDD sample 
(Hagedorn et al. 2008). Estimating the denominator is not an issue for the Bias Study because the 
residential status of each sampled address is known. 

Two sets of unit response rates were computed for the Bias Study: the reduced treatment 
response rates and the full treatment response rates. In the reduced treatment, the Telephone Research 
Center (TRC) status is treated as the final household status for Bias Study cases attempted in the TRC, 
and the field status is treated as the final household status for nonmatch and mismatch cases. In the full 
treatment, the final household status is based on efforts in both the TRC and in the field for cases 
attempted in the TRC, and the field status is the final household status for nonmatch and mismatch cases. 
For example, a case that was a final refusal in the TRC but responded to the Screener in the field is treated 
as a nonrespondent in the reduced treatment but as a respondent in the full treatment. Exhibit 5-1 shows 
how cases were classified for the reduced and full treatments. Table 5-1 shows the disposition of the 
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7,500 Bias Study cases under both the reduced treatment and the full treatment. After the full treatment, 
92 percent of the Bias Study cases were identified as residential. A total of 4,894 cases were respondents 
with the full treatment, and 4,235 were respondents under the reduced treatment. Table 5-2 gives the 
weighted and unweighted Screener unit response rates for the Bias Study and, by way of comparison, for 
the RDD sample. The Screener unit response rates for the RDD sample, the Bias Study full treatment, and 
the Bias Study reduced treatment are 52.8 percent, 67.7 percent, and 59.8 percent, respectively. Because 
the RDD sample is restricted to households with telephones whereas the Bias Study sample involves both 
telephone and nontelephone households whose addresses appear on the U.S. Postal Service residential 
delivery file, the comparable groups between the two samples are the RDD mailable cases and the Bias 
Study address-telephone matched cases. The Screener unit response rate for the Bias Study address- 
telephone matched cases is 59.9 percent, compared to 54.4 percent for the RDD mailable sample. 

Exhibit 5-1. Screener response status classification for the NHES:2007 Bias Study sample 



Screener response status^ 



Attempted in TRC* 


Screener 
completed 
in TRC 


Found to be 
nonresidential 
in field 


Screener 
completed 
in field 


Reduced 

treatment 


Full 

treatment 


Number of 
cases 


Yes 


Yes 


t 


t 


R 


R 


2,235 


Yes 


No 


Yes 


t 


NR 


I 


63 


Yes 


No 


No 


Yes 


NR 


R 


659 


Yes 


No 


No 


No 


NR 


NR 


857 


No 


t 


Yes 


t 


I 


I 


441 


No 


t 


No 


Yes^ 


R 


R 


2,000 


No 


t 


No 


No 


NR 


NR 


1,245 



t Not applicable. 

* Cases not attempted in the TRC are those with no matching telephone number or a mismatched telephone number. 
^R: respondent; NR: nonrespondent; I: ineligible. 

^ Includes the 109 potential mismatches and 1 RDD duplicate case. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 
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Table 5-1. Number of Screener cases attempted, by residential and response status, and weighted 
percentages of cases: NHES:2007 Bias Study 



Screener response 
category 


Reduced treatment 




Full treatment 




Number 

(unweighted) 


Percent 

(weighted) 


Number 

(unweighted) 


Percent 

(weighted) 


Total 


7,500 


100.0 


7,500 


100.0 


Residential 


7,059 


93.0 


6,996 


92.3 


Responded 


4,235 


55.6 


4,894 


62.5 


Did not respond 


2,824 


37.4 


2,102 


29.8 


Nonresidential 


441 


7.0 


504 


7.7 



NOTE: The numbers of cases are unweighted counts; the percentages are weighted. Detail may not sum to totals 
because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 



Table 5-2. Weighted and unweighted Screener unit response rates for the RDD sample, the RDD 
mailable cases only, the Bias Study sample, and the Bias Study address-telephone 
matched only: NHES:2007 RDD and NHES:2007 Bias Study 



Overall unit response rates 



Weighted (percent) Unweighted (percent) 



RDD sample 


52.8 


53.2 


RDD mailable' 


54.4 


55.0 


Bias Study sample, reduced treatment 

Bias Study sample, reduced treatment, address- 


59.8 


60.5 


telephone matched only 


59.9 


60.3 


Bias Study sample, full treatment 


67.7 


70.0 



' The unweighted rate was computed using the Wave 1 sample only, and assuming 37.6 percent of telephone 
numbers with an unknown residency status (no answer and no answer, answering machine cases) are residential. 
See section 4.2.1 of Hagedom et al. (2008) for details. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program, 2007. 



Table 5-3 shows the numbers of screened households in which household members were 
sampled for extended interviews by the sample to which the cases belonged and the extended interviews 
completed. In the Bias Study sample with full treatment, about 59 percent of households had at least one 
household member sampled for an extended interview. The distribution of sampled extended interviews 
in the households under reduced treatment is comparable to the full treatment distribution. The RDD 
sample distribution of sampled extended interviews differs from the Bias Study distribution because for 
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the RDD sample the sampling of adults for AEWR interviews was done only in the Wave 1 sample (a 
random subsample comprising approximately 54 percent of the lull RDD sample). 



Profile of Screener Unit Response Rates 



In most RDD surveys, it is difficult to obtain and examine the characteristics of households 
that do not respond to the screening interview. Consequently, the ability to examine nonresponse bias at 
this stage of the survey is limited. In table 5-4, the Screener unit response rates are given by 
characteristics of the geographic area of the households^® based on the 2000 Census, and by whether an 
answering machine message was left during the study. These characteristics were considered because they 
are available for all cases and have the potential to be associated with response propensity. 

Table 5-3. Number and percent of households responding to the Screener, by sample type and 
type of extended interviews scheduled: NHES:2007 RDD and NHES:2007 Bias Study 





RDD sample 


Bias Study reduced 
treatment 


Bias Study full 
treatment 


Type of interview scheduled 


Number of 
households 


Percent of 
households 


Number of 
households 


Percent of 
households 


Number of 
households 


Percent of 
households 


Total 


54,034 


100.0 


4,235 


100.0 


4,894 


100.0 


At least one extended interview 


24,725 


45.8 


2,452 


57.9 


2,882 


58.9 


School Readiness (SR) 


interview only 

Parent and Family Involvement 


1,305 


2.4 


118 


2.8 


133 


2.7 


in Education (PFI) interview only 


9,846 


18.2 


782 


18.5 


936 


19.1 


SR and PFI interview 
Adult Education for Work- 
Related Reasons (AEWR) 


1,988 


3.7 


165 


3.9 


204 


4.2 


interview only 


9,157 


16.9 


1,081 


25.5 


1,245 


25.4 


PFI and AEWR interviews 


2,016 


3.7 


248 


5.9 


290 


5.9 


SR and AEWR interviews 


242 


0.4 


37 


0.9 


45 


0.9 


SR, PFI, and AEWR interviews 


171 


0.3 


21 


0.5 


29 


0.6 


No extended interview 


29,309 


54.2 


1,783 


42.1 


2,012 


41.1 



NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program, 2007. 



The subsampling into waves was done to allow for more efficient and effective use of follow-up procedures in a study with a short data 
collection period such as NHES. Cases in the Wave 1 sample were followed up more intensively than the remainder of the sample. See Hagedom 
et al. (2008) for further details. 

The sampling of adults for AEWR interviews was discontinued after Wave 1 due to low response rates. 

For the RDD sample, these are characteristics of the ZIP code that has the most households associated with telephone numbers in the exchange; 
for the Bias Study sample, these are characteristics of the predominant ZIP code in the block. 
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The response rate patterns by area characteristic for the Bias Study (both the full treatment 
and the reduced treatment) were compared to those observed in the RDD study. The one characteristic, 
shown in table 5-4, that stands out is the indicator of whether an answering machine message was left. In 
the RDD sample and in the Bias Study reduced treatment, the Screener response rate was higher among 
households in which no answering machine message was left. In the Bias Study full treatment, the rate 
was higher among households in which at least one answering machine message was left. Among Bias 
Study cases (with the full treatment), the Screener unit response rate varied by region of the country, with 
the highest unit response rate in the Midwest and the lowest unit response rate in the West. Areas with 
higher proportions of Whites generally had higher unit response rates than those with lower proportions 
of Whites, and correspondingly, areas with lower proportions of Hispanics, Blacks, and Asians had higher 
unit response rates than those with higher proportions in these subgroups. Areas with lower median home 
values generally had higher unit response rates than those with higher median home values. Areas with 
higher proportions of renters had lower response rates than those with lower proportions of renters. In 
general, these response rate patterns for the Bias Study (both the full treatment and the reduced treatment) 
are consistent with those observed in the RDD study. 

This univariate profde of Screener unit response rates by the characteristics of the areas is 
difficult to interpret because there are so many characteristics to consider. In addition, some of these 
characteristics are correlated, and the univariate profile does not explore these relationships. 
Consequently, a multivariate analysis, which is discussed in chapter 6, was performed to examine the 
interrelationship of the characteristics and the Screener unit response rates. 



5.3 Extended Interview Unit Response Rates and Overall Unit Response Rates 

During the screening interview, all children were enumerated in households with eligible 
children; adults were enumerated in only a subsample of households. After the enumeration, children or 
adults within the household were sampled for the SR, PFI, or AEWR surveys. The person who was 
identified as the most knowledgeable about the sampled child’s care and education (nearly always a 
parent and most often the child’s mother) was designated to be the respondent for the SR or PFI 
interview. The AEWR interview was conducted with the sampled adult. 

The number of persons enumerated and sampled, and those with completed interviews for 
each survey are given in table 5-5; the table includes counts for both the RDD sample and the Bias Study 
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(full treatment) sample. Table 5-6 gives the unit response rates and overall unit response rates for the 
RDD sample and the Bias Study (full treatment) sample. For the SR survey, the unit response rate is 79.8 
percent for the Bias Study sample, compared to 77.0 percent for the RDD sample; for the PFI survey the 
unit response rates are 75.9 percent for the Bias Study sample and 74.1 percent for the RDD sample; for 
the AEWR survey, the unit response rates are 60.6 percent and 62.4 percent for the Bias Study and RDD 
samples, respectively. 



Table 5-4. Weighted Screener unit response rates for the NHES:2007 RDD sample, reduced 

treatment NHES:2007 Bias Study sample, and full treatment NHES:2007 Bias Study 
sample, by selected characteristics 



Characteristic 


RDD sample 


Bias Study sample, 
reduced treatment 


Bias Study sample, 
full treatment 


Total 


52.8 


59.8 


67.7 


Mailable status 


Mailable address 


54.4 


59.8 


67.7 


No mailable address 


43.3 


t 


t 


Answering machine message indicator 


No message left 


65.7 


64.0 


67.0 


One or more messages left 


42.3 


48.7 


69.4 


Percent White 


Less than 30 percent 


43.4 


46.1 


51.5 


30 to 39 percent 


46.1 


55.2 


62.4 


40 to 59 percent 


47.8 


57.3 


64.5 


60 to 69 percent 


49.7 


56.6 


63.3 


70 to 79 percent 


52.8 


63.4 


71.6 


80 to 89 percent 


53.6 


61.3 


69.2 


90 percent or more 


58.0 


63.1 


71.9 


Percent Black 


0 to 49 percent 


53.0 


60.0 


67.9 


50 percent or more 


48.3 


51.5 


60.3 


Percent Asian 


Less than 10 percent 


54.7 


61.9 


70.1 


10 to 19 percerrt 


47.6 


53.0 


59.0 


20 to 29 percent 


44.4 


44.8 


52.6 


30 percent or more 


39.2 


49.1 


55.7 


Percent Hispanic 


0 to 39 percent 


53.4 


60.8 


68.9 


40 percent or more 


44.7 


49.2 


54.6 



See notes at end of table. 
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Table 5-4. Weighted Screener unit response rates for the NHES:2007 RDD sample, reduced 

treatment NHES:2007 Bias Study sample, and full treatment NHES:2007 Bias Study 
sample, by selected characteristics — Continued 



Characteristic 


RDD sample 


Bias Study sample, 
reduced treatment 


Bias Study sample, 
full treatment 


Median home value 


U‘ decile 


57.6 


66.7 


74.2 


2"“^ through 4* deciles 


57.0 


64.1 


72.2 


5* through 6* deciles 


54.3 


60.0 


68.8 


7* through 9* deciles 


49.3 


55.3 


62.7 


10* decile 


44.2 


53.0 


60.1 


Percent renters 


0 to 49 percent 


53.8 


61.0 


69.1 


50 percent or more 


42.3 


51.1 


57.0 


Percent college graduates 


Less than 20 percent 


54.2 


62.9 


70.7 


20 to 29 percent 


54.0 


59.0 


66.4 


30 percent or more 


51.3 


58.2 


66.4 


Household income decile 


Less than 40 percent 


54.3 


61.4 


69.0 


40 to 59 percent 


53.6 


61.4 


69.7 


60 percent or more 


54.5 


57.4 


65.3 


Census region 


Northeast 


49.9 


58.1 


67.1 


Midwest 


59.3 


66.7 


74.2 


South 


52.6 


58.9 


67.1 


West 


49.2 


57.2 


63.9 


Census division 


New England 


51.9 


57.6 


66.7 


Middle Atlantic 


49.1 


58.5 


67.4 


East North Central 


57.3 


68.2 


76.6 


West North Central 


64.0 


63.8 


69.5 


South Atlantic 


51.6 


55.9 


63.9 


East South Central 


56.7 


70.7 


82.2 


West South Central 


52.3 


57.0 


63.0 


Mountain 


55.8 


70.1 


79.8 


Pacific 


46.5 


52.8 


58.5 



t Not applicable. 

NOTE: The categories of each characteristic are based on a multivariate analysis for the nonresponse weighting 
adjustment, in which the sample was divided into subgroups with the most differential response rates. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program, 2007. 
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Table 5-5. Number of enumerated children and adults, by type of extended interview and final 
extended interview status: NHES:2007 RDD sample; NHES:2007 Bias Study sample, 
full treatment 



Type of interview 


RDD sample 


Bias Study sample, full treatment 


Sehool Readiness (SR) interview 






Enumerated 


4,030 


450 


Sampled 


3,706 


411 


Ineligible 


239 


39 


Did not respond 


735 


75 


Total eomplete 


2,633 


299 


Parent and Family Involvement in 






Edueation (PFI) interview 
Enumerated 


23,882 


2,480 


Sampled 


14,021 


1,459 


Ineligible 


92 


6 


Did not respond 


3,347 


335 


Total eomplete 


10,681 


1,115 


Adult Edueation for Work-Related 






Reasons (AEWR) interview 
Enumerated 


31,314 


4,276 


Sampled 


11,586 


1,609 


Ineligible 


236 


40 


Did not respond 


3,640 


504 


Complete 


7,710 


1,065 



NOTE: Counts given in this table are based on elassifieations aeeording to how the person was sampled. For 



example, if a ehild was sampled for the SR interview but was found to be eligible for the PFI interview instead, that 
ehild is ineluded in the SR interview eounts in this table. 

SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, Sehool Readiness (SR) Survey of 
the National Flousehold Edueation Surveys Program, 2007; Parent and Family Involvement in Edueation (PFI) Survey 
of the National Flousehold Edueation Surveys Program, 2007; and Adult Edueation for Work-Related Reasons 
(AEWR) Survey of the National Flousehold Edueation Surveys Program, 2007. 

Table 5-6. Weighted unit response rates and overall unit response rates, by type of extended 
interview: NHES:2007 RDD sample; NHES:2007 Bias Study sample, full treatment 



Type of interview 


RDD sample 


Bias Study sample, full treatment 


Unit response rate 


Overall unit 
response rate 


Unit response rate 


Overall unit 
response rate 


SR interview 


77.0 


40.7 


79.8 




PFI interview 


74.1 


39.1 


75.9 


51.4 


AEWR interview 


62.4 


33.0 


60.6 





NOTE: The overall unit response rate is the produet of these unit response rates for the surveys and the Sereener unit 
response rates of 52.8 pereent for the RDD sample and 67.7 pereent for the full treatment Bias Study sample. 



SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, Sehool Readiness (SR) Survey 
of the National Flousehold Edueation Surveys Program, 2007; Parent and Family Involvement in Edueation (PFI) 
Survey of the National Flousehold Edueation Surveys Program, 2007; and Adult Edueation for Work-Related 
Reasons (AEWR) Survey of the National Flousehold Edueation Surveys Program, 2007. 
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5.4 



Item Response Rates 



In the SR, PFI, and AEWR Surveys of NHES:2007, as in most surveys, the responses to 
some data items are not obtained for all interviews. There are numerous reasons for item nonresponse. 
Some respondents do not know the answer for the item or do not wish to respond for other reasons. Some 
item nonresponse arises when an interview is interrupted and not continued later, leaving items at the end 
of the interview blank. Item nonresponse may also be encountered when responses provided by the 
respondent are not internally consistent, and this inconsistency is not discovered until after the interview 
is completed. In these cases, the items that were not internally consistent were set to missing and then 
imputed. 



For items on the SR, PFI, and AEWR^’ surveys, the median item response rates for the Bias 
Study were 99.3 percent, 99.4 percent, and 99.8 percent, respectively. These rates are comparable to the 
corresponding item response rates for the RDD sample (99.3 percent, 99.0 percent, and 99.7 percent, 
respectively). 

As in the main NHES: 2007 study, most items on the SR, PFI, and AEWR surveys had item 
response rates over 90 percent. For the SR survey, there were two items for which the response rate was 
substantially^^ lower in the Bias Study than for the RDD sample (SEDOWEEE, the number of times since 
the beginning of the school year that the child’s teacher/school has contacted the household about 
anything the child is doing particularly well or better in preschool, and SEENJOY, the extent to which the 
child enjoys school) and two items in the PFI survey (SEBEHAV, the number of times since the 
beginning of the school year that the child’s teacher/school has contacted the household about any 
behavior problems the child is having in school, and SESCHOE, whether the family applied for a 
scholarship or grant for the child). There is no reason to believe that SEDOWEEE, SEENJOY, 
SEBEHAV, and SESCHOE are sensitive items that would be subject to lower response rates with the in- 
person effort than with strictly telephone collection. Thus, there is no indication of systematic differences 
between the two samples in the willingness or ability of respondents to respond to items. In summary, the 
item response rates for the NHES:2007 surveys, from both the RDD sample and the Bias Study sample, 
are high and consistent across samples. 



For the AEWR survey, the median item response rates given correspond to the set of items that were imputed. For this survey, because no 
public-use data file was produced, only a subset of items needed for the analysis were imputed. 

In this context, “substantially” is defined as a difference of 5.0 percent or more between the RDD item response rate and the Bias Study item 
response rate, for items that were applicable to more than 20 cases in each of the samples. 
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For analysis purposes, numeric and categorical data items with missing data were imputed. 
The imputations were done because complete responses were needed for the variables used in developing 
the sampling weights and for the variables needed for this analysis; additionally, for the RDD sample, the 
items were fully imputed in order to provide data users with data fdes containing complete cases. 

A hot-deck procedure was used to impute most missing responses. In this approach, the 
entire fde was sorted into cells defined by characteristics of households or respondents that are likely to 
be associated with differences in item response propensities. These characteristics, or boundary variables, 
were used to group respondents into those most likely to have the same response or the same response 
propensity for the data item to be imputed. The hot-deck procedure has been used for imputation in all 
previous NHES administrations, and the specific procedures used for imputation for the NHES:2007 
surveys were based on the procedures used in prior surveys. For details on the hot-deck procedure used 
for NHES:2007, see Hagedom et al. (2008). 
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6. WEIGHTING 



The objective of the National Household Education Surveys Program (NHES:2007) surveys 
is to make inferences about the entire civilian, noninstitutionalized population for the domains of interest. 
Weighting is necessary to account for differential probabilities of selection and to reduce potential bias 
due to nonresponse and differential coverage of subpopulations. However, weighting is only effective in 
reducing bias to the extent that the auxiliary variables used in weighting are related to the outcomes of 
interest. Even after weighting adjustments are made, some bias, if it exists, is likely to remain. The 
purpose of the bias analysis is to assess the bias in the final weighted estimates from the NHES:2007, to 
the extent possible. 

To accomplish this, three sets of weights were used in the bias analysis. The first set 
corresponds to the main random digit dialing (RDD) sample and is described in Hagedom et al. (2008).^^ 
This set does not contain any of the Bias Study sample. The other two sets were created for the Bias 
Study analysis. The weights differ in their treatment of Bias Study sample households that were coded as 
nonrespondents in the Telephone Research Center (TRC) before being sent to the field. As described in 
chapter 5, the reduced treatment method treats the status in the TRC as final, and the frill treatment 
method assigns the status based on the entire TRC and field effort. For all three sets of weights, the 
estimates were adjusted to totals of persons living in both telephone and non-telephone households so the 
estimates represent the same population. 

The weighting process for the two sets of Bias Study sample weights are described below. 



6.1 Household-Level Weights 

The primary purpose of the Screener in NHES:2007 was to provide information required to 
assess the eligibility of household members for an extended interview. Household-level information that 
is of analytic interest was also collected during the extended interview. Since no data intended for 
analyses were collected at the household level only, as with the RDD sample, household-level weights 
were calculated solely to provide general characteristics of the Bias Study sample and for use as a basis 
for computing person-level weights for the analysis of the extended interview data. 

The weighting procedures used for NHES:2007 are similar to those used in NHES:2005. (See Hagedom et al. 2006.) 
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The area sample for the Bias Study was designed to be an equal probability sample of 
addresses within phone match status. The household- level weight for the Bias Study sample was thus 
the product of three factors: 

■ weight associated with the differential sampling of addresses with a telephone number 
match and those without {Ajy, 

■ adjustment for Screener nonresponse (Cy); and 

■ poststratification adjustment to align estimates with external population totals {Ej). 



The procedures for computing the Bias Study household-level weights follow. The 
procedure for the lull treatment and reduced treatment weights differ only in the assignment of response 
status in step 2. 

1 . Addresses with a telephone number match were sampled at twice the rate of addresses 
without a matching telephone number. Therefore, the household-level base weight, Aj, 
was set equal to 1 for addresses with a telephone number match and 2 for addresses 
without a match. 

2. The second weighting factor adjusts for households that did not respond to the 
NHES:2007 Screener. Each household in the NHES Bias Study sample was classified 
as either a respondent (R), a nonrespondent (NR), or an ineligible case (I). The 
classification differed for the full treatment and reduced treatment methods, as was 
shown in exhibit 5-1. 

The base weights of the nonrespondent cases were distributed to the base weights of 
the respondent cases within a nonresponse adjustment cell. A Chi-Square Automatic 
Interaction Detection (CHAID) analysis was used to identify characteristics most 
associated with Screener nonresponse. (For a description of the CHAID analysis, refer 
to section 6.3.) These characteristics, which were primarily geographic characteristics 
associated with the ZIP code, were used to form the cells for nonresponse adjustment 
of the household weights. The same set of characteristics was considered for the Bias 
Study as for the RDD sample. One additional variable, whether the address had a 
valid matching phone number, was also a potential candidate. This variable was not 
relevant for the RDD sample, but was shown to be an important predictor of 
nonresponse in the Bias Study sample. All other variables identified by CHAID as 
important predictors of Screener nonresponse for the Bias Study sample were also 
identified as important predictors for the RDD sample. 



The PSUs and segments within PSUs were sampled such that all addresses ultimately had the same probability of selection, hence there is no 
weighting adjustment factor accounting for that level of sample development. 

Characteristics used in household nonresponse adjustment for the Bias Study sample included whether the household had a telephone number 
match, census division, percentage White in the ZIP Code, median home value in the ZIP Code, and whether an answering machine message was 
ever left. 
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The same nonresponse adjustment cells were used for both sets of Bias Study weights. 
Table 6-1 contains the cells used for Screener nonresponse adjustment in the 
NHES:2007 Bias Study, along with the estimated Screener unit response rate for each 
cell under both the full treatment and reduced treatment. The nonresponse adjustment 
factor, Cj(c), applied to each responding household j in adjustment cell c is 



Tw = 



heR^.uNR^ 

heR, 



where Af^ is the weight for household h associated with differential sampling of 
addresses with a telephone number match and those without, and R^, and are the 
sets of respondents and nonrespondents in cell c, respectively. Note that these sets of 
respondents and nonrespondents are defined differently for the full treatment and 
reduced treatment methods, according to exhibit 5-1. 

3. The final step in computing the household weight was to adjust to known national 
control totals in order to account for household-level over- or under-coverage in the 
address lists used for sampling and to enable the production of estimates for the RDD 
sample, full treatment Bias Study sample, and reduced treatment Bias Study sample 
that represent the same population. Poststratification was used to accomplish this task. 
Poststratification adjusts survey weights to known population totals. The 
characteristics used in poststratification were census region and presence of children 
under 18 years of age. Table 6-2 presents the control totals used for poststratifying the 
household-level weights. The variables used in poststratification of the household 
weights were the same for the Bias Study sample as for the RDD sample; for the RDD 
sample, these variables were chosen to address differences in landline telephone 
coverage rates with respect to region in which the household is located and presence 
of children in the household. The control totals for poststratification were obtained 
from the March 2006 CPS. 

The final household-level weight for household j, HHWj, is given by 

where Ej(d) is the poststratification adjustment factor described above for adjustment 
cell d, where household j has the attributes corresponding to poststratification cell d. 
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Table 6-1. Screener nonresponse adjustment cells: NHES:2007 Bias Study 



Cell 


Nonmatching 
or mismatched 
telephone number 


Census 

division 


Percent 

White 


Median 
home value 


Answering 
machine 
message left 


Full treatment 
estimated 
response rate 
(percent)' 


Reduced treatment 
estimated response 
rate (percent)^ 


1 


1 


1,5,9 


0,1, 2,3 ,4,5,6 


t 


t 


48 


48 


2 


1 


1,5,9 


7 


0,1,2,3,4 


t 


79 


79 


3 


1 


1,5,9 


7 


5,6,7,8,9 


t 


59 


59 


4 


1 


1,5,9 


8,9 


0,1,2,3,4,5 


t 


53 


53 


5 


1 


1,5,9 


8,9 


6,7,8, 9 


t 


54 


54 


6 


1 


2,4 


t 


0 


t 


79 


79 


7 


1 


7 


t 


0 


t 


63 


63 


8 


1 


2,4,7 


t 


1 


t 


51 


51 


9 


1 


2,4,7 


t 


2,3,4 


t 


68 


68 


10 


1 


2,4,7 


t 


5,6,7,8,9 


t 


48 


48 


11 


1 


3 


t 


t 


t 


71 


71 


12 


1 


6,8 


t 


t 


t 


78 


78 


13 


2 


1,2,3,6,8 


t 


0,1,2,3,4,5 


1 


81 


53 


14 


2 


4,5,7,9 


t 


0,1,2,3,4,5 


1 


70 


45 


15 


2 


t 


t 


6,7,8,9 


1 


66 


45 


16 


2 


t 


0,1,2 


t 


2 


64 


54 


17 


2 


t 


3,4,5,6,7 


0,1,2 


2 


83 


70 


18 


2 


t 


3,4,5,6,7 


3,4,5 


2 


94 


76 


19 


2 


t 


3,4,5,6,7 


6,7,8,9 


2 


77 


65 


20 


2 


t 


8,9 


t 


2 


88 


78 



t Not applicable. In these situations, either no cases in the cell had the condition or the cell consisted of all values of 
the particular variable. 

* The estimated response rate is the number of completed interviews divided by the sum of the number of completed 
interviews and nonresponses, weighted by the probability of selection. Cases found in the field to be nonresidential 
are considered ineligible and are excluded. 

^ The estimated response rate for cells with a nonmatching or mismatched telephone number is the same as in the full 
treatment method. For cells with a matching phone number (nonmatching or mismatched telephone number = 2), the 
estimated response rate is the number of completed interviews in the TRC, divided by the number of cases attempted 
in the TRC, weighted by the probability of selection. 

NOTE: Category codes were as follows: Nonmatching or mismatched telephone number: 1 = no match or incorrect 
match; 2 = match. Census Division: 0 = Alaska and Flawaii; 1 = New England; 2 = Middle Atlantic; 3 = East North 
Central; 4 = West North Central; 5 = South Atlantic; 6 = East South Central; 7 = West South Central; 8 = Mountain; 
9 = Pacific (excluding Alaska and Flawaii). Percent White: 0 = less than 10 percent, 1 = 10 to 19 percent, 2 = 20 to 
29 percent, 3 30 to 39 percent, 4 = 40 to 49 percent, 5 = 50 to 59 percent, 6 = 60 to 69 percent, 7 = 70 to 79 percent, 
8 = 80 to 89 percent, 9 = 90 percent or more. Median home value: 0 = below the 10th percentile in sample, 1 = 10th to 
19th percentile in sample, 2 = 20th to 29th percentile in sample, 3 = 30th to 39th percentile in sample, 4 = 40th to 49th 
percentile in sample, 5 = 50th to 59th percentile in sample, 6 = 60th to 69th percentile in sample, 7 = 70th to 79th 
percentile in sample, 8 = 80th to 89th percentile in sample, 9 = 90th percentile in sample or higher. Answering machine 
message left: 1 = yes; 2 = no. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Flousehold Education 
Surveys Program (NHES), 2007. 
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Table 6-2. Control totals for poststratifying the NHES:2007 household-level weights: CPS:2006 



Census region' 


Control totaP 


Total 


114,510,050 


Northeast 

No ehildren under 1 8 in household 
Children under 1 8 in household 


13,993,709 

7,137,051 


South 

No ehildren under 1 8 in household 
Children under 1 8 in household 


27,173,229 

14,638,867 


Midwest 

No ehildren under 18 in household 
Children under 18 in household 


17,390,279 

8,981,331 


West 

No ehildren under 18 in household 
Children under 18 in household 


15,731,203 

9,464,380 



The following states and the Distriet of Columbia are in eaeh eensus region: Northeast: CT, MA, ME, NH, NJ, NY, 
PA, RI, VT; South: AL, AR, DC, DE, FL, GA, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA, WV; Midwest: lA, 
IL, IN, KS, MI, MN, MO, ND, NE, OH, SD, WI; West: AK, AZ, CA, CO, HI, ID, MT, NV, NM, OR, UT, WA, 



WY. 

^ The eontrol totals are numbers of households. 

NOTE: Detail may not sum to totals beeause of rounding. 

SOURCE: U.S. Department of Commeree, Bureau of the Census, Current Population Survey (CPS), Mareh 2006. 



6.2 Person-Level Weights for the School Readiness (SR), Parent and Family Involvement 

in Education (PEI), and Adult Education for Work-Related Reasons (AEWR) 
Interviews 

As described in chapter 3, a sampling algorithm was used to limit the number of persons 
sampled in each household while maintaining the sampling rates required to attain the target sample sizes. 
The sampling was based on information collected in the Screener interview from the adult household 
member who responded to the Screener. For the SR and PFI Surveys, the eligibility of the sampled child 
was later verified or updated when the parent/guardian most knowledgeable about the child responded to 
the SR or PFI interview, provided that person was not the Screener respondent. For the AEWR Survey, an 
eligible adult was defined to be a person 16 years of age or older who was not enrolled in grade 12 or 
below, not institutionalized, and not on active duty in the U.S. Armed Forces. Because sampling 
eligibility was defined in terms of the data collected in the Screener, the weighting procedures were 
developed with possible misclassification (i.e., children sampled for the SR survey who were found to be 
eligible for the PFI survey and vice versa; adults sampled for AEWR as participants who were found to be 
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nonparticipants and vice versa) taken into account so that the estimates would not incur bias due to 
misclassification. 



The household-level weight was used as the base weight for each of the person-level (e.g., 
SR, PFI, and AEWR interview) weights. The person-level weight for sampled person k in household j, 
PWjk, is the product of the household weight, HHWj, and four weight adjustment factors: 



■ weight associated with sampling the person’s domain in the given household (Ajk); 

■ weight associated with sampling the person from among all eligible persons in the 
given domain in the household (Bjk); 

■ weight associated with extended interview (SR, PFI or AEWR) unit nonresponse 
(Cjk); and 



■ adjustment associated with raking^® the person-level weights to Census Bureau 
estimates of the number of persons in the target population (Djk). 

The same procedures were used to compute the person-level weight adjustments in the Bias 
Study sample as in the RDD sample. The weighting steps for the full treatment and reduced treatment 
methods were also the same, but the input household-level weights and set of sampled persons differed 
based on the classification of the household in step 2 of the household-level weighting process. The steps 
for the person-level weights are described below. 

1. The first step in developing the person-level weights was to account for the 
probability of sampling the person’s domain in the given household. For both SR and 
PFI, if there was an eligible child in the household, then one child was selected for the 
survey. Thus, the factor for sampling in both the SR and PFI domain was always equal 
to 1. 

Exhibit 6- 1 gives the weighting factors, Ajk, used to account for the probability of 
sampling the adult domains for AEWR, based on the household composition. Note 
that the domain probabilities of selection are given in table 3-1. For example, if there 
were no eligible children in the household and there were two eligible adults — one 
adult education participant and one adult education nonparticipant — then the adult 
education participant was sampled with probability 0.3637 and the adult education 
nonparticipant was sampled with probability 0.1819. In such an example, if the adult 
education participant was sampled, then the weighting factor Ajt for that adult was 
2.7493, which is the reciprocal of the probability of sampling the adult domain. If the 
adult education nonparticipant was sampled, then the weighting factor Ajk was 5.4985. 



See step 4 below for a definition and detailed discussion of raking. 
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Exhibit 6-1. Weighting factors to account for domain sampling for adults: NHES:2007 



Number of SR 
eligible 
children in 
household 


Number of PFI 
eligible 
children in 
household 


Number of adults in household, by 
adult education participation status 


Weighting factor associated with 
domain sampling 


Adult education 
participant 


Adult education 
nonparticipant 


Adult education 
participant 


Adult education 
nonparticipant 


0 


0 


0 


1 or more 


— 


3.6657 


0 


0 


1 or more 


0 


1.8328 


— 


0 


0 


1 or more 


1 or more 


2.7493 


5.4985 


0 


1 or more 


0 


1 or more 


— 


6.8249 


0 


1 or more 


1 or more 


0 


3.4125 


— 


0 


1 or more 


1 or more 


1 or more 


5.1187 


10.2374 


1 or more 


0 


0 


1 or more 


— 


6.8249 


1 or more 


0 


1 or more 


0 


3.4125 


— 


1 or more 


0 


1 or more 


1 or more 


5.1187 


10.2374 


1 or more 


1 or more 


0 


1 or more 


— 


14.6628 


1 or more 


1 or more 


1 or more 


0 


7.3314 


— 


1 or more 


1 or more 


1 or more 


1 or more 


10.9971 


21.9941 



— Indicates that factor is not applicable because there are no adults in the given domain in the household. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 



2. The second adjustment, which accounted for the probability of sampling person k 
from among all eligible persons in the given domain in household j, is 

Bjk = Njk , 

where Njk is the number of persons in household j in the same sampling domain as 
person k. 

For each sampled person jk, the unadjusted person-level weight, UPWjk, can be 
written as the product of the household-level weight and the adjustments for within- 
household sampling. That is, for sampled person jk, the unadjusted person-level 
weight is 

UPWjk=HHWjk-Ajk-Bjk. 

3. The next step was to adjust for persons (most knowledgeable parents/guardians in the 
case of the SR and PFI interviews, and the sampled adults themselves in the case of 
the AEWR interview) who did not respond to the extended interview. Each extended 
interview case was classified as either a respondent (R) or a nonrespondent (NR), 
depending on whether or not the extended interview was completed for the sampled 
person. The unadjusted person-level weights (UPW) of the nonrespondents were 
distributed to the unadjusted person-level weights of the respondents within a 
nonresponse adjustment cell. For the SR and PFI Surveys, the nonresponse adjustment 
cells were created using a home ownership indicator and grade (where enrolled 
children with no grade equivalent were included in the cell containing the modal grade 
for their age; that is, they were assigned to the grade in which most children their age 
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are enrolled). These variables were used because they are available for all sampled 
children (both respondents and nonrespondents) and are associated with SR/PFI 
interview response propensity. See table 6-3 for a list of SR/PFI nonresponse 
adjustment cells. For the RDD sample, Census region and age/grade combinations 
were used to form the SR/PFI nonresponse adjustment cells. 

Table 6-3. School Readiness/Parent and Family Involvement in Education (SR/PFI)-NHES:2007 
interview nonresponse adjustment cells: NHES:2007 Bias Study 



Explanatory variable (Home ownership/ 
Grade or equivalent from Screener) 


Number of 
respondents in 
cell for full 
treatment 


Completion 
rate for full 
treatment 
(percent) 


Number of 
respondents in 
cell for reduced 
treatment 


Completion 
rate for reduced 
treatment 
(percent) 


Own/all grades 

Rent or other/Unenrolled or preschool 


1,038 


79.4 


873 


81.0 


through 8 th grade 


304 


74.1 


270 


75.0 


Rent or other/9th through 12th grade 


73 


57.5 


61 


56.2 



SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, Sehool Readiness (SR) Survey 
of the National Household Edueation Surveys Program, 2007. 



For the AEWR interview, three variables were used to create the nonresponse 
adjustment cells. The first was an indicator of whether the sampled adult was the 
Screener respondent, the second was the adult education participation status of the 
adult (as reported by the Screener respondent), and the third was the sex of the adult. 
The variables were the same as with the RDD sample, but some collapsing of 
categories was necessary because of smaller sample sizes. These variables were used 
because they are available for all sampled adults (both respondents and 
nonrespondents) and are associated with AEWR interview response propensity. (See 
table 6-4 for a list of the AEWR interview nonresponse adjustment cells.) The 
nonresponse adjustment factor, Cjk(c), applied to each respondent jk in adjustment cell 
c is 



Y,UPW„ 

.-y h^R^'<jNR^ 



where UPWj^ is the unadjusted person-level weight for sampled person h. Thus, for 

each sampled person jk, the nonresponse-adjusted person-level weight, NPWjk, can be 
written as 



NPWjk =UPWjk-Cjk(c). 

Extreme weights may occasionally result when households or persons are sampled at 
very different rates. Additionally, the procedures used for nonresponse adjustment and 
poststratification may contribute to extreme weights. A few unexpectedly large 
sampling weights can seriously inflate the variance of the survey estimates. Thus, for 
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a small number of records, weight trimming procedures may be used to reduce the 
impact of such large weights on the estimates produced from the sample. Weight 
trimming refers to the process of artificially adjusting a few extreme weights (those 
that are unusually large relative to other weights for members of the same subgroup) 
to reduce their impact on the weighted estimates. 

Table 6-4. Adult Education for Work-Related Reasons (AEWR)-NHES:2007 interview 
nonresponse adjustment cells: NHES:2007 Bias Study 



Explanatory variables (Indicator of 
whether the sampled adult was the 
Screener respondent/adult education 
participation status from Screener/sex) 


Number of 
respondents in 
cell for full 
treatment 


Completion 
rate for full 
treatment 
(percent) 


Number of 
respondents in 
cell for reduced 
treatment 


Completion 
rate for reduced 
treatment 
(percent) 


Screener respondent/adult education 
participant/male 


118 


81.7 


103 


81.5 


Screener respondent/adult education 
participant/ female 


273 


77.9 


237 


78.2 


Screener respondent/adult education 
nonparticipant/ male 


145 


79.2 


129 


81.1 


Screener respondent/adult education 
nonparticipant/female 


222 


82.1 


189 


82.5 


Not Screener respondent/both 

participation statuses/both sexes 


307 


40.4 


273 


40.7 



SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, National Household Edueation 
Surveys Program, 2007. 



The variability in the nonresponse adjusted person-level weights was examined by 
population subgroups to determine whether trimming would be desirable. For the SR, 
PFI, and AEWR person-level weights, there was not enough variability to justify 
trimming. 

4. The final stage of person-level weighting involved raking the nonresponse-adjusted 
person-level weights, NPW, to national control totals. Raking was proposed by 
Doming and Stephan (1940) as a way to ensure consistency between complete counts 
and sample data from the 1940 U.S. Census of population. The raking procedure 
typically improves the reliability of survey estimates, and also corrects for the bias 
due to persons not covered by the survey. Additionally, raking provides the ability to 
generate population estimates that match external estimates, in particular for the Bias 
Study, to generate population estimates comparable to the RDD sample. The raking 
procedure is carried out in a sequence of adjustments: first, the base weights are 
adjusted to one marginal distribution (or dimension) and then the second marginal 
distribution, and so on. One sequence of adjustments to the marginal distributions is 
known as a cycle or iteration. The procedure is repeated until convergence of 
weighted totals to all sets of marginal distributions is achieved. (See Doming and 
Stephan 1940 for further details on raking and the convergence process.) 

This additional raking adjustment, following the household-level poststratification 
adjustment, is required because the extended interviews involve new eligibility criteria 
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and a new level of sampling. That is, although the household-level poststratification 
adjustment aligned the weighted totals of the household weights with the household- 
level control totals, the raking of the person-level weights is required in order to align 
the person-level weights with the person-level control totals and adjust for differential 
coverage rates at the person level. 

The raking procedure for the SR and PFI weights involved raking the nonresponse- 
adjusted person-level weights to national totals obtained using percentage 
distributions from the October 2005 Current Population Survey (CPS) and the total 
number of children from the March 2006 CPS. The October 2005 CPS contains 
variables not available on the March 2006 CPS, but the totals in the latter are more 
current. The control total for a raking cell is the proportion in that cell from the 
October 2005 CPS multiplied by the estimate of the total number of children from the 
March 2006 CPS. 

The four raking dimensions used for the SR interview weights were race/ethnicity of 
the child (Black, non-FIispanic/Hispanic/other), household income categories 
($25,000 or less/$25,001 or more), a cross of census region (Northeast/South/ 
Midwest/West) and urbanicity (urban/rural), and a cross of home tenure (rent/own or 
other) and grade of child (with those enrolled in school but having no grade equivalent 
assigned to the modal grade for their age). The three raking dimensions used for the 
PFI interview weights were a cross between race/ethnicity of the child (Black, non- 
Hispanic/FIispanic/other) and household income categories ($10,000 or less/$ 10,00 1- 
$25,000/$25,001 or more), a cross of census region (Northeast/South/Midwest/West) 
and urbanicity (urban/rural), and a cross of home tenure (rent/own or other) and grade 
of child (with those enrolled in school but having no grade equivalent assigned to the 
modal grade for their age). These raking variables were the same ones used in the 
RDD sample weighting, but with some collapsing of categories because of smaller 
sample sizes. This collapsing of categories was done in order to make the raking 
adjustment more stable. In general this would be expected to result in less variation in 
the weights and, therefore, a decrease in variances, relative to what the variances 
would have been if collapsing had not been done. For the RDD survey, these raking 
dimensions were used because they include important analysis variables (e.g., grade) 
and characteristics that have been shown to be associated with telephone coverage 
(e.g., race/ethnicity) (Blumberg and Luke 2006). Tables 6-5 and 6-6 show the control 
totals used for raking the SR and PFI interview weights, respectively. 

For the AEWR interview, the four dimensions for the raking cells were a cross of the 
adult’s race/ethnicity (Black, non-Hispanic/Hispanic/other) and household income 
($10,000 or less/$10,001-$25,000/$25,001 or more), a cross of age (16-29 years/30- 
49 years/50 years or more) and sex, a cross of census region (Northeast/South/ 
Midwest/West) and urbanicity (urban/rural), and a cross of home tenure (rent/own or 
other) and highest educational attainment (less than high school diploma/high school 
diploma or equivalent/some college). These raking dimensions were generally the 
same ones used for the RDD sample; however, the cross of age, sex, and household 
size was replaced with age by sex, and collapsing across regions in rural areas. The 
collapsing of categories was necessary because of smaller sample sizes for the Bias 
Study sample. The elimination of household size as a raking variable for the Bias 
Study sample is justifiable, since it was used in the RDD weighting to account for any 




noncoverage of single, young males based on evidence of higher rates of cell-phone 
only households among this group, which is not an issue in the area sample. 



Table 6-5. Control totals for raking the School Readiness (SR)-NHES:2007 person-level interview 
weights: NHES:2007 Bias Study 



Total 




8,734,486 


Raee/ethnieity of ehild 




Control totah 


Blaek, non-Hispanie 




1,319,184 


Hispanie 




1,918,622 


Other 




5,496,680 


Household ineome 




Control totah 


$25,000 or less 




2,331,634 


$25,001 or more 




6,402,852 


Census region' 


Urbanieity 


Control totah 


Northeast 


All 


1,417,809 


South 


Urban 


2,301,791 


South 


Rural 


858,488 


Midwest 


Urban 


1,539,271 


Midwest 


Rural 


521,193 


West 


All 


2,095,934 


Home tenure 


Age/grade of ehild 


Control totah 


Rent 


Unenrolled 


1,714,544 


Rent 


Presehool 


1,350,853 


Own or other 


Unenrolled 


2,429,177 


Own or other 


Presehool 


3,239,912 



' The following states and the Distriet of Columbia are in eaeh eensus region: Northeast: CT, MA, ME, NH, NJ, 
NY, PA, RI, VT; South: AL, AR, DC, DE, FL, GA, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA, WV; Midwest: 
lA, IE, IN, KS, MI, MN, MO, ND, NE, OH, SD, WI; West: AK, AZ, CA, CO, HI, ID, MT, NV, NM, OR, UT, WA, 



WY. 

^ The eontrol totals are numbers of people. 

NOTE: Detail may not sum to totals beeause of rounding. 

SOURCE: U.S. Department of Commeree, Bureau of the Census, Current Population Survey, Mareh 2006 and 
Oetober 2005. 
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Table 6-6. Control totals for raking the Parent and Family Involvement in Education (PFI)- 
NHES:2007 person-level interview weights: NHES:2007 Bias Study 



Total 




53,185,978 


Race/ethnicity of child 


Household income 


Control totaP 


Black, non-Hispanic 


$10,000 or less 


1,672,661 


Black, non-Hispanic 


$10,001-$25,000 


1,998,302 


Black, non-Hispanic 


$25,001 or more 


4,226,716 


Hispanic 


$10,000 or less 


952,408 


Hispanic 


$10,001-$25,000 


2,735,705 


Hispanic 


$25,001 or more 


6,240,396 


Other 


$10,000 or less 


1,416,805 


Other 


$10,001-$25,000 


3,411,705 


Other 


$25,001 or more 


30,531,280 


Census region' 


Urbanicity 


Control total^ 


Northeast 


Urban 


8,046,784 


Northeast 


Rural 


1,488,933 


South 


Urban 


13,986,891 


South 


Rural 


5,216,625 


Midwest 


Urban 


8,749,595 


Midwest 


Rural 


2,962,588 


West 


Both urbanicities 


12,734,562 


Home tenure 


Grade of child 


Control total^ 


Rent 


Transitional kindergarten/kindergarten/pre- 1st grade 


1,186,672 


Rent 


1st grade 


1,328,422 


Rent 


2nd grade 


1,166,716 


Rent 


3rd grade 


1,216,608 


Rent 


4th grade 


1,082,817 


Rent 


5 th grade 


1,105,484 


Rent 


6th grade 


1,045,527 


Rent 


7 th grade 


1,045,227 


Rent 


8 th grade 


1,124,203 


Rent 


9th grade 


1,113,763 


Rent 


10 th grade 


1,081,231 


Rent 


1 1th grade 


918,851 


Rent 


12th grade 


757,865 


Own or other 


Transitional kindergarten/kindergarten/pre- 1st grade 


2,715,226 


Own or other 


1st grade 


2,806,353 


Own or other 


2nd grade 


2,750,847 


Own or other 


3rd grade 


2,698,323 


Own or other 


4th grade 


2,767,402 


Own or other 


5 th grade 


2,941,790 


Own or other 


6th grade 


3,007,403 


Own or other 


7 th grade 


3,097,426 



See notes at end of table. 
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Table 6-6. Control totals for raking the Parent and Family Involvement in Education (PFI)- 
NHES:2007 person-level interview weights: NHES:2007 Bias Study — Continued 



Home tenure — Continued 


Age/grade of child — Continued 


Control totaf 


Own or other 


8th grade 


3,115,756 


Own or other 


9th grade 


3,157,928 


Own or other 


10 th grade 


3,283,568 


Own or other 


1 1th grade 


3,496,583 


Own or other 


12th grade 


3,173,987 



' The following states and the Distriet of Columbia are in eaeh eensus region: Northeast: CT, MA, ME, NH, NJ, NY, 
PA, RI, VT; South: AL, AR, DC, DE, FL, GA, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA, WV; Midwest: lA, 
IL, IN, KS, MI, MN, MO, ND, NE, OH, SD, WI; West: AK, AZ, CA, CO, HI, ID, MT, NV, NM, OR, UT, WA, 



WY. 

^ The eontrol totals are numbers of people. 

NOTE: Detail may not sum to totals beeause of rounding. 

SOURCE: U.S. Department of Commeree, Bureau of the Census, Current Population Survey, Mareh 2006 and 
Oetober 2005. 



The control totals for raking the AEWR interview weights, shown in table 6-7, were 
obtained from the March 2006 CPS. The raking iterations were continued until the 
estimated totals were within 1 of all the control totals. 

The final person-level weight for each sampled person jk is 

PWjk =NPWjk 

where Djk(d) is the raking adjustment factor for raking cell d, where person jk has the 
attributes corresponding to the levels of the dimensions of raking cell d. 
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Table 6-7. Control totals for raking the Adult Education for Work-Related Reasons (AEWR)- 
NHES:2007 person-level weights: NHES:2007 Bias Study 



Total 

Race/ ethnicity 

Black, non-Hispanic 

Black, non-Hispanic 

Black, non-Hispanic 

Hispanic 

Hispanic 

Hispanic 

Other 

Other 

Other 

Age 

16-29 years 
16-29 years 
30-49 years 
30-49 years 
50 years or more 
50 years or more 

Urbanicity 

Urban 

Urban 

Urban 

Urban 

Rural 

Home tenure 



Household income 
$10,000 or less 
$10,001-$25,000 
$25,001 or more 
$10,000 or less 
$10,001-$25,000 
$25,001 or more 
$10,000 or less 
$10,001-$25,000 

$25,001 or more 

Sex 

Male 

Female 

Male 

Female 

Male 

Female 

Census region' 

Northeast 

South 

Midwest 

West 

All regions 

Educational attainment 



216,827,342 
Control totaP 
3,040,804 
5,143,163 
16,137,645 
1,827,866 
5,398,828 
20,753,194 
7,481,096 
21,434,761 
135,609,985 
Control total^ 
22,799,648 
22,825,035 
41,617,822 
42,862,494 
39,941,394 
46,780,949 
Control total^ 
34,452,602 
57,024,617 
36,023,610 
43,867,176 
45,459,337 
Control total^ 



Rent Less than high school diploma 13,341,302 

Rent High school diploma or equivalent 30,037,210 

Rent Some college 14,598,715 

Own or other Less than high school diploma 1 8,474, 1 60 

Own or other High school diploma or equivalent 8 1 , 1 42,050 

Own or other Some college 59,233,905 



The following states and the District of Columbia are in each census region: Northeast: CT, MA, ME, NH, NJ, 
NY, PA, RI, VT; South: AL, AR, DC, DE, FL, GA, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA, WV; Midwest: 
lA, IE, IN, KS, MI, MN, MO, ND, NE, OH, SD, WI; West: AK, AZ, CA, CO, HI, ID, MT, NV, NM, OR, UT, WA, 
WY. 

^ The control totals are numbers of people. 

NOTE: Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Commerce, Bureau of the Census, Current Population Survey, March 2006. 
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6.3 



Chi-Square Automatic Interaction Detection (CHAID) Analysis 



As mentioned in section 6.2, Screener nonresponse adjustment cells for weighting were 
formed based on the results from an analysis used to identify characteristics most associated with 
Screener nonresponse. A univariate profile of Screener unit response rates by the characteristics of the 
geographic areas, as presented in chapter 5, is difficult to interpret because there are many characteristics 
and some of the characteristics are correlated. In order to study the interrelationships among 
characteristics and the Screener unit response rate, a multivariate analysis was conducted. The goal of the 
multivariate analysis was to better understand the complex relationships among the characteristics by 
examining the characteristics simultaneously with regard to unit response rates, and to determine if 
groups of households had extremely different unit response rates. Nonresponse bias in the estimates may 
appear when the characteristics of the respondents and nonrespondents are different. (See chapter 2 for a 
more detailed discussion of the relationship between response propensities and nonresponse bias.) By 
identifying groups with different unit response rates, the characteristics of the respondents and 
nonrespondents can be used as an indicator of the potential for nonresponse bias, and thus using these 
characteristics to form cells for nonresponse adjustment may reduce nonresponse bias (Little 1986). 

The characteristics of the geographic areas corresponding to the sampled addresses were 
used to identify groups with different unit response rates. The multivariate analysis was done using a 
categorical search algorithm called Chi-Square Automatic Interaction Detection (CHAID). This algorithm 
is similar to the continuous search algorithms LISREL and Automatic Interaction Detector (AID) that 
have been used for a number of years, but it is designed especially to handle categorical data like those 
available for the Bias Study sample. CHAID first identifies the characteristic of the data that is the best 
predictor of response. Then, within the levels of that characteristic, CHAID identifies the next most likely 
response predictor(s), and so forth, until a tree is formed with all potential response predictors. The final 
result is a division of the entire dataset into cells by attempting to determine sequentially the cells that 
have the greatest discrimination with respect to the unit response rates. In other words, it divides the 
dataset into groups so that the unit response rate within cells is as constant as possible, and the unit 
response rate between cells is as different as possible. This automatic procedure was done by specifying 
that the minimum number of households in any group had to be greater than or equal to 1 00 and the split 
of the variables into subgroups had to be statistically significant using a chi-square test at the 95 percent 
significance level. 
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Since many of the variables in the CHAID model, such as median home value, have multiple 
response categories, the program must take this into account. The CHAID software does this in two ways. 
First, it allows the dataset to be split into subgroups separately within each level of the characteristic 
chosen in the previous round of CHAID selection. For example, census division categories are grouped 
differently within each of the phone number match categories. (For an example, see table 6-1.) Second, 
the procedure selects variables irrespective of the number of response categories that variable may have 
since the procedure collapses categories together to get meaningful categories. 

All of the characteristics in the model are tested, and the one with the response categories 
having the largest discrimination with respect to the unit response rates is identified.” Table 6-1 contains 
the summary of this analysis as it relates directly to weighting the data. In this case, telephone number 
match was the variable chosen as most associated with response propensity. Among cases without a 
telephone number match, census division was identified as the characteristic next-most-associated with 
response propensity. Among cases with a telephone number match, whether an answering machine 
message was left was identified as the characteristic next-most-associated with response propensity. The 
process of identifying the characteristic most associated with Screener response propensity, conditional 
on the characteristics already identified, continued until the final 20 cells shown in table 6-1 were formed. 
In addition to telephone number match status, census division, and the answering machine message 
indicator, the final 20 cells were formed using percent White in the ZIP code and median home value in 
the ZIP code. Although the ZIP code-level variables (i.e., percent Black, percent Asian, percent Hispanic, 
percent renters, college graduates and household income decile) were considered in the CHAID analysis, 
they were not selected as discriminators of response propensity in this multivariate analysis, given the 
other characteristics. The range of unit response rates among some of the cells suggests that interactions 
among some characteristics may be present. For example, for cells 1 through 5 (phone number matched 
cases in the New England, South Atlantic, or Pacific census divisions), the Screener unit response rates 
range from 48 to 79 percent. 



^’Variables identified in previous analyses as being associated with response propensity were selected from among the variables available for 
both responding and nonresponding units. For the Bias Study, little information is available for nonresponding households, limiting the selection 
of characteristics for the CHAID analysis. Information associated with key characteristics of interest, such as participation in early childhood 
programs, activities, or adult education, and correlates of these, such as maternal employment or educational attainment, are not available for 
nonrespondents, and therefore cannot be used for nonresponse adjustment purposes. 
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The range of unit response rates among the 20 cells suggested that the key characteristics 
identified by CHAID should be used in creating weighting adjustments. As a result, these 20 cells were 
used in the adjustment for Screener nonresponse. These results suggest that the weighting adjusts for 
some of the important characteristics associated with nonresponse bias. 



6.4 Sampling Errors 

In surveys with complex sample designs, such as NHES:2007, direct estimates of the 
sampling errors assuming a simple random sample will typically underestimate the variability in the 
estimates (Wolter 1985). The NHES:2007 Bias Study sample design and estimation included procedures 
that deviate from the assumption of simple random sampling, such as oversampling addresses with a 
phone number match, sampling persons within households with differential probabilities, and raking to 
control totals. 

One method for computing sampling errors to reflect these aspects of the sample design and 
estimation is the replication method. Replication involves splitting the entire sample into a set of groups 
or replicates based on the actual sample design of the survey. The survey estimates can then be computed 
for each of the replicates by creating replicate weights that mimic the actual sample design and estimation 
procedures used in the lull sample. The variation in the estimates computed from the replicate weights 
can then be used to estimate the sampling errors of the estimates from the frill sample. The replication 
method was used to produce the standard errors of estimates for the bias analysis. All standard error 
calculations were performed using the WesVar software. 

A total of 80 replicates were defined for NHES:2007 Bias Study sample. This number was 
chosen to match the number of replicates for the RDD survey. The specific replication procedure used for 
NHES:2007 Bias Study was the jackknife (JKn) replication method (Wolter 1985).^* It involved dividing 
the sample into 80 random subsamples (replicates) for the computation of the replicate weights. The first 
39 replicates were formed based on the sample design, and the remaining 41 replicates were “pseudo” 
replicates in which the full sample weights were used. The extra 4 1 replicates do not contribute anything 
to the variance but were created to simplify variance calculations. In each replicate, a replicate weight was 
developed using the same weighting procedures that were used to develop the full sample weight. 



The jackknife (JKl) replication method was used for the RDD sample weights. A description can be found in Hagedom et al. (2008). 
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Replicate weights were created for each of the NHES:2007 surveys: the SR, the PFI, and the 
AEWR. The procedures for forming the Bias Study replicate weights for each of these surveys are 
described below. The procedure for the RDD sample can be found in Hagedom et al (2008).^^ 

1. Variance strata and variance units were formed to reflect the sample design. For the 
29 noncertainty PSUs, variance strata were defined as the strata used for sampling 
(see section 3.2), which contained 2 or 3 sampled PSUs (variance units) per stratum. 
For the 1 certainty PSU, 5 variance strata were formed by pairing sampled segments 
(variance units) within the PSU. 

2. The first 39 replicates were created using the JKn method. The replicate 1 base 
weights were assigned by multiplying the full-sample base weight for the first 
variance unit in the first variance stratum by zero and the other variance units in the 
first variance stratum by a factor of «/,/(«;, ~ 1)> where riu is the number of variance 
units in the variance stratum (equal to 2 for the first variance stratum). The full sample 
weights for all variance units in the remaining variance strata were multiplied by a 
factor of 1 . This process was repeated for each of the remaining variance units to form 
the rest of the 39 replicates. The weights for replicates 40 through 80 were set equal to 
the full sample weights. 

3. Using the exact same weighting procedures described earlier in this chapter for each 
of the sets of full sample weights, the other adjustments (i.e., sampling adjustments, 
nonresponse adjustments, and raking adjustments) were applied to every replicate 
base weight for completed interviews. In other words, the weighting steps were 
applied 80 times. 

4. The difference in the methods used for the full sample and for the replicate weights 
was that the raking iterations were stopped when the replicate weights converged to 
within 10 of the control totals rather than 1, which was used in the full sample 
weighting. 

The replication procedure for the NHES:2007 surveys involves the calculation of 81 
estimates, including an estimate using the full sample weight and estimates using each of the 80 replicate 
weights. 



The JKn variance estimator, vl^ I , has the form 





The procedures for forming replicates for the NHES:2007 RDD sample are the same as the procedures used in NHES:2005. (See Hagedom et 
al. 2006.) 
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where 6 is the population parameter of interest; 6 is the estimate of 0 based on the full sample; is 

the estimate of 6 based on the observations included in the replicate; and rik is number of variance 
units in the variance stratum corresponding to replicate k. 



6.5 Significance Tests 

All differences discussed in this report are significant at the 95 percent confidence level, 
based on a 2-sided t-test. While some relatively small differences (3 to 5 percentage points) might be 
statistically significant when sample sizes are large, the discussion is limited to differences that are 
potentially of substantive importance. Differences of substantive importance are defined as differences of 
5 percentage points or more or relative differences of 3 or more (i.e., when one estimate is 3 or more 
times larger than the other). The Bias Study was designed to allow detection of a 5 percentage point 
difference in key statistics. For NFIES, this is considered a meaningful threshold to use to identify which 
statistically significant differences are of substantive significance. 

When the comparison involves correlated samples, the standard error in the t-test was 
calculated to appropriately account for the correlation. An example of a comparison with correlated 
samples is the nonresponse bias analysis, in which the bias is the difference between the estimate from the 
reduced treatment method, which includes only Bias Study respondents finalized in the TRC or with no 
matching telephone number or a mismatched number, and the full treatment method, which includes all 
Bias Study respondents. 
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7. AN OVERVIEW OF BIAS 



This chapter provides an overview of the bias in the NHES:2007 estimates. Section 7.1 
provides some methodological issues to consider when comparing the estimates from the NHES:2007 
Bias Study sample to estimates from other sources. The estimates from the Bias Study sample are 
intended to give an indication of the bias in the random digit dialing (RDD) sample estimates. However, 
the Bias Study estimates are themselves potentially subject to bias. Section 7.2 compares the estimates 
from the Bias Study sample to those from an external source as a check of the reasonableness of the Bias 
Study estimates; the external source used for this comparison is the Current Population Survey (CPS). 
The Bias Study sample and RDD sample estimates are computed using weights adjusted for nonresponse 
and raked to population totals, as described in chapter 6. To evaluate the effect of the weighting 
adjustments on bias in the RDD estimates, section 7.3 contains a comparison of NHES:2007 RDD 
estimates before and after the adjustments. In section 7.4, the Bias Study estimates are compared to the 
final, adjusted estimates from the NHES:2007 RDD sample. Differences reflect the overall bias in the 
RDD sample estimates, including nonresponse and noncoverage bias. A breakdown of the components of 
bias is given in chapter 8. 



7.1 Methodological Considerations in Data Comparisons 

Sample and nonsampling errors, sample sizes, methods of survey administration, the timing 
of surveys, and response rates all affect the data collected and any comparisons made (Bradburn 1983; 
Groves 1989). In addition, question wording variation, question order, question context, and respondent 
recall can have a major impact on survey responses (Bradburn 1983; Groves 1989). While comparisons of 
the Bias Study estimates and RDD sample estimates are intended to give an indication of the noncoverage 
and nonresponse bias in the RDD estimates, the differences might also be attributable to noncoverage or 
nonresponse in the Bias Study sample as well as other sources of sampling or nonsampling error. Similar 
issues are important when comparing the Bias Study estimates to external sources. As a result, it is 
important to note some general methodological issues. 

Every survey, including the NHES:2007 Bias Study, is subject to both sampling error and 
nonsampling error. Sampling errors occur because the data are collected from a sample rather than a 
census of the population. Because the sample of addresses selected for the Bias Study is just one of the 
many possible samples that could have been selected, estimates produced from the Bias Study sample 
may differ from estimates that would have been produced from other samples. In the same way, the data 
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from the NHES:2007 RDD sample and the CPS are also subject to sampling error. Nonsampling errors 
are errors made in the collection and processing of data and may be caused by population coverage 
limitations and data collection, processing, and reporting procedures. The sources of nonsampling error 
are typically problems like unit and item nonresponse, the differences in respondents’ interpretations of 
the meaning of the questions, response differences related to the particular time the survey was 
conducted, and mistakes in data preparation. Although the NHES surveys are designed to account for 
sampling error and minimize nonsampling error, the estimates presented in this chapter are subject to both 
types of error. These types of errors are not unique to NHES, but are common to all sample surveys. 

Population coverage is an issue that arises in the examination of results of any telephone 
survey because households without telephones are excluded from the sample. The NHES:2007 RDD data, 
which were obtained from a sample of residential landline telephone numbers, were statistically adjusted 
to reduce the effects of population noncoverage due to lack of telephone ownership. As a result, the 
estimates from RDD sample sum to the total number of eligible persons in all households, not just those 
in households with telephones. Although these statistical adjustments may be useful in reducing biases in 
aggregates for the whole population, more serious biases may exist for estimates of segments of the 
population with relatively low telephone coverage rates (Brick, Burke, and West 1992). Unlike the RDD 
sample, the Bias Study sample includes households without telephones. However, it is still subject to 
population noncoverage. The address lists used for sampling have been shown to provide poorer coverage 
of rural areas (O’Muircheartaigh, Eckman, and Weiss 2002). The Bias Study data were statistically 
adjusted to reduce the effects of population noncoverage (see chapter 6), but some noncoverage bias 
might still remain. 

Timing of survey administration in terms of the years in which surveys were conducted or 
the time of year they were administered also may affect responses (Groves 1989). The time of the year 
when the data are collected can affect responses to questions related to specific topics such as school 
attendance. It is important to keep in mind that the data collection period can be an important factor to 
consider when comparing Bias Study and CPS estimates. The Telephone Research Center (TRC) data 
collection periods for the NHES:2007 Bias Study sample and RDD sample were the same. 

Variation in response rates across surveys can also result in differences in the estimates. To 
the extent that nonrespondents are different from respondents, low response rates may introduce biases 
into the survey estimates. In the Bias Study, the overall unit response rate was 54.0 for School Readiness 
(SR), 51.4 for Parent and Family Involvement in Education (PEI), and 41.0 for Adult Education for 
Work-Related Reasons (AEWR). For the RDD sample, the overall unit response rate was 40.7 percent for 
SR, 39.1 percent for PEI, and 33.0 percent for AEWR. These response rates are given in table 5-6, and 
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unit response rates for NHES:2007 are discussed more thoroughly in chapter 5. Unit response rates for the 
comparable data sources discussed in this chapter were 83.3 percent for CPS March 2006 and 89.5 
percent for CPS October 2005. 

The mode of administration (e.g. telephone interview versus face-to-face interview) is 
another factor related to responses (Groves 1989). For example, differences in mode can affect question 
wording, question context, or the interviewer-respondent interaction. Interviews were conducted by 
telephone for the NHES RDD sample, the Bias Study reduced treatment, and the Bias Study full 
treatment (see chapter 4). Therefore, differences in these estimates can not be attributed to mode effects. 

Because NHES data are adjusted with a raking procedure to match CPS population totals, 
the Bias Study and RDD sample estimates exactly match CPS estimates for the characteristics used in the 
raking, provided the categorization is the same as that used in raking. Because the standard error of an 
estimate is a measure of sampling error variance, a standard error of 0 indicates the absence of sampling 
error variance. When NHES estimates of totals are adjusted to exactly match CPS totals (through the 
raking adjustment), all sampling error in those estimated totals is eliminated, under the assumption that 
the CPS total is the true population value. Any NHES estimate of a characteristic not specifically 
controlled for in the raking adjustment would not be expected to exactly match CPS totals for one or more 
of the reasons discussed earlier in this section. 



7.1.1 General Comments on the Comparisons 

The estimates to be presented here are just some of the multitude of comparisons that could 
be made between NHES:2007 Bias Study estimates and the RDD sample and CPS estimates using 
different variables and categorizations of those variables. The items included in these comparisons were 
selected because they include important characteristics of persons and households, or because they are 
key outcome variables from the surveys. When many comparisons are made, some will undoubtedly show 
statistically significant differences. The main purpose of the comparisons is to explore the overall quality 
of the data and to determine whether there are some differences in estimates of substantive importance 
that need to be investigated further. 
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7.1.2 



Other Data Considerations 



As is true for most surveys, responses were not obtained for all the NHES:2007 data items 
for all interviews. Despite the high item response rate, all NHES:2007 missing data items were imputed. 
The median Bias Study item response rates for items in the SR, PEI, and AEWR surveys were 99.3, 99.4, 
and 99.8 percent, respectively."'® The CPS estimates provided as comparison data also contain imputed 
data. 



Another data consideration is age. The CPS includes respondents ages 15 and older, whereas 
Adult Education for Work-Related Reasons (AEWR)-NHES:2007 Bias Study adults were at least 16 
years old. For the purpose of the comparisons pertaining to adults in this chapter, this difference in the age 
subgroup was accounted for by restricting tabulations of the CPS data to persons ages 16 and older. 



7.2 Comparison of NHES:2007 Bias Study Estimates to an External Source 

This section presents a comparison for selected estimates from the NHES:2007 Bias Study 
with estimates from the CPS. In the bias analysis presented in section 7.4, estimates from the full Bias 
Study will be used as the standard by which to evaluate bias in the RDD estimates, since the response 
rates for the Bias Study are higher than for the main study RDD sample, and undercoverage bias from the 
address-based sampling frame is believed to be less of an issue than undercoverage from the RDD 
sampling frame. The comparison in this section is intended to provide an indication of the reasonableness 
of the selected NHES:2007 Bias Study estimates for this purpose. The CPS was selected for this 
comparative analysis because it included topical information and samples comparable to those used in the 
NHES:2007 surveys. Historically, the CPS has been used as the extant source to which NHES 
demographic estimates have been compared. For these comparisons, the SR, PEI, and AEWR Bias Study 
estimates were calculated using adjusted weights. 



7.2.1 Current Population Survey (CPS) 

The CPS is a monthly household survey conducted by the Bureau of the Census to provide 
information about employment, unemployment, and other characteristics of the civilian 
noninstitutionalized population. The CPS respondent is a household member age 15 or older and the 



* The median RDD Study item response rates for items in the SR, PFI, and AEWR surveys were 99.3, 99.0, and 99.7 percent, respectively. 
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survey is conducted each month with a sample of approximately 72,000 households located in 754 
primary sampling units. The U.S. Department of Education is a joint sponsor of the annual October 
supplement to the CPS, which provides specific information on educational topics. 

CPS data from October 2005 and March 2006"*' were used for comparison with estimates 
from SR, PFI, and AEWR surveys. The October 2005 supplement contains the most recently available 
CPS data regarding enrollment status and grade by type of school in which students are enrolled, and the 
March 2006 supplement contains the most recent CPS data on age, race/ethnicity by educational 
attainment, industry, and occupation. The October 2005 CPS data were collected on 71,270 households 
and 137,809 individuals; and the March 2006 CPS data were on 71,700 households and 135,028 
individuals. The data comparisons below for SR, PFI, and AEWR cover key estimates including ages of 
subject, student grade, enrollment status, school type, sex, and highest level of educational attainment. 

The CPS public-use data fdes do not contain the information required to compute standard 
errors directly. However, the CPS provides documentation on computing approximate standard errors 
using generalized variance functions (GVFs). GVFs are functions that model the variance (or standard 
error) of survey estimates based on the value of the estimates. Further information on the CPS GVFs can 
be found on the CPS website, at http://www.census.gov/cps . The GVFs were used to obtain approximate 
standard errors for each of the CPS estimates presented in this chapter. 



7.2.2 Comparability of the NHES:2007 Bias Study and CPS Distributions 

Age of persons. Table 7-1 shows NHES:2007 Bias Study and 2005 CPS estimates of the age 
distribution of the population as indicated by the age of persons who were subjects of NHES interviews 
(i.e., children/youth from ages 3 to 20 and enrolled in grade 12 or below and noninstitutionalized adults 
ages 16 or older and not enrolled in grade 12 or below). All observed differences are 1 percentage point or 
less, with estimates not exhibiting statistically significant differences when applying 95 percent 
confidence intervals. 

School type and student grade level. Estimates of the number of children enrolled in 
kindergarten through grade 12, by school type and by student grade level, are presented in table 7-2 for 
the Bias Study PFI survey and for CPS:2005. Estimates of the number of children at each grade level 
from kindergarten through grade 12 are not significantly different. Number estimates are rounded to the 

The October 2005 and March 2006 CPS data were the most recent available at the time this report was drafted. Generally, the CPS shows little 
variation over 1- and 2-year time spans. 
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nearest thousand for ease of interpretation. The NHES:2007 Bias Study estimates show that there were 
53,186,000 children enrolled in kindergarten through grade 12, and the CPS:2005 estimates show that 
there were 53,328,000 children (a difference of 142,000 children or 0.3 percent of the NHES estimate). 
The percentage distributions for grade are nearly identical between NHES:2007 and CPS:2005 because 
grade was used for raking. The numbers of children enrolled in public and private school are also 
comparable. 



Table 7-1. Percentage distribution for age of subjects of interviews: SR-NHES:2007 Bias Study, 
PFI-NHES:2007 Bias Study, AEWR-NHES:2007 Bias Study, and CPS:2005 



Age category 


SR-NHES:2007 Bias Study, PFl- 
NHES:2007 Bias Study and AEWR- 
NHES:2007 Bias Study' 


CPS:2005 




Percent 


S.e. 


Percent 


S.e. 


3-5 years 


4 


0.1 


4 


0.1 


6-9 years 


6 


0.1 


6 


0.1 


10-15 years 


9 


0.2 


9 


0.1 


16-19 years 


5 


0.6 


6 


0.1 


20-29 years 


15 


0.5 


14 


0.1 


30-39 years 


13 


0.9 


14 


0.1 


40-49 years 


17 


0.9 


16 


0.1 


50-59 years 


14 


1.0 


14 


0.1 


60 or more years 


17 


1.0 


17 


0.1 



' Estimates of children ages 3 through 6 and not yet enrolled in kindergarten were obtained from the School 
Readiness (SR) Survey. Estimates of children/youth ages 3 through 20 and enrolled in kindergarten through grade 
12 were obtained from the Parent and Family Involvement in Education (PFl) Survey. Estimates of adults ages 16 
and older, not enrolled in grade 12 or below, and not on activity duty in the U.S. Armed Forces were obtained from 
the Adult Education for Work-Related Reasons (AEWR) Survey. Parent respondents to the SR and PFl Surveys are 
not included in calculations for adult estimates. 

NOTE: s.e. is standard error. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, School Readiness Survey of the 
National Household Education Surveys Program (NHES), 2007; Parent and Family Involvement in Education 
Survey of the NHES, 2007; and Adult Education for Work-Related Reasons Survey of the NHES, 2007. U.S. 
Department of Commerce, Bureau of the Census, Current Population Survey, October 2005. 
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Table 7-2. Number of children in kindergarten through grade 12, by school type and by student 
grade level: PFI-NHES:2007 Bias Study and CPS:2005 





PFI-NHES:2007 Bias Study 


CPS:2005 


School type and grade 


Number 

(thousands) 


S.e. 

(thousands) 


Number 

(thousands) 


S.e. 

(thousands) 


Total number of children in kindergarten 


through 12th grade 


53,186 


0 


53,328 


330 


School type' 


Public 


46,867 


801 


48,018 


320 


Private 


5,269 


868 


5,309 


124 


Homeschooled 


1,050 


273 


— 


— 


Student grade level 


Kindergarten 


3,902 


0 


3,912 


107 


1 


4,135 


0 


4,146 


no 


2 


3,918 


0 


3,928 


107 


3 


3,950 


40 


3,925 


107 


4 


3,850 


0 


3,860 


106 


5 


4,012 


40 


4,058 


109 


6 


4,053 


0 


4,064 


109 


7 


4,143 


0 


4,154 


no 


8 


4,240 


0 


4,251 


111 


9 


4,272 


0 


4,283 


112 


10 


4,365 


0 


4,376 


113 


11 


4,415 


0 


4,427 


113 


12 


3,932 


0 


3,942 


107 



' CPS does not isolate homesehoolers, some of whom may also be attending sehool for a eertain number of hours 
per week. 



NOTE: s.e. is standard error. Detail may not sum to totals beeause of rounding. 

SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, Parent and Family Involvement 
in Edueation Survey of the National Household Edueation Surveys Program, 2007; and U.S. Department of 
Commeree, Bureau of the Census, Current Population Survey, Oetober 2005. 



Table 7-3 shows estimates of the number of children enrolled in kindergarten through grade 
12 in public versus private schools. There are no statistically significant differences of 5 percentage points 
or more between PFI-NHES:2007 Bias Study and CPS:2005 with respect to enrollment in public and 
private schools across grade levels. 
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Table 7-3. Number and percentage of children in kindergarten through grade 12 enrolled in 
public and private schools: PFI-NHES:2007 Bias Study and CPS:2005 









School type 










Public 






Private 




Child’s current grade 


Number 

(thousands) 


Percent 


S.e. 


Number 

(thousands) 


Percent 


S.e. 


PFI-NHES:2007 Bias 
Study 
K 


3,204 


84 


5.9 


611 


16 


5.9 


1,2 


7,129 


92 


1.9 


611 


8 


1.9 


3,4 


7,103 


94 


2.1 


438 


6 


2.1 


5, 6 


6,861 


88 


3.5 


966 


12 


3.5 


7,8 


7,538 


90 


3.9 


844 


10 


3.9 


9, 10 


7,519 


88 


3.1 


1,039 


12 


3.1 


11, 12 


7,514 


91 


3.2 


760 


9 


3.2 


CPS:2005 


K 


3,349 


86 


1.0 


563 


14 


1.0 


1,2 


7,153 


89 


0.6 


921 


11 


0.6 


3,4 


7,031 


90 


0.6 


755 


10 


0.6 


5, 6 


7,270 


90 


0.6 


852 


10 


0.6 


7,8 


7,574 


90 


0.6 


831 


10 


0.6 


9, 10 


7,967 


92 


0.5 


692 


8 


0.5 


11, 12 


7,675 


92 


0.5 


695 


8 


0.5 



NOTE: s.e. is standard error. For the National Household Education Surveys Program: 2007, kindergarten (K) 
includes grades reported as kindergarten, transitional kindergarten, and prefirst grade. Children who are 
homeschooled are not included. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement 
in Education Survey of the National Household Education Surveys Program, 2003; and U.S. Department of 
Commerce, Bureau of the Census, Current Population Survey, October 2005. 



Household income: SR. Table 7-4 presents Bias Study SR and CPS estimates of the 
percentage of children ages 3 through 6, not yet enrolled in kindergarten who resided in households with 
particular income categories. Across income categories, estimates from both surveys are comparable. 
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Table 7-4. Percentage of children ages 3 through 6 not yet enrolled in kindergarten, by household 
income: SR-NHES:2007 Bias Study and CPS:2005 



Household income 


SR-NHES:2007 Bias Study 


CPS:2005 




Percent 


S.e. 


Percent 


S.e. 


$15,000 or less 


15 


1.7 


15 


0.7 


$15,001 to $30,000 


18 


2.4 


18 


0.8 


$30,001 to $50,000 


18 


2.8 


21 


0.8 


Over $50,000 


50 


2.9 


47 


1.0 



NOTE: s.e. is standard error. Current Population Survey estimates exelude eases with missing ineome data. Beeause 
of rounding, pereentages may not add to 100. 

SOURCE: U.S. Department of Edueation, National Center for Edueation Statisties, Sehool Readiness Survey of the 
National Household Edueation Surveys Program, 2007; and U.S. Department of Commeree, Bureau of the Census, 
Current Population Survey, Oetober 2005. 



Household income by race/ethnicity: SR. Table 7-5 compares SR-NHES:2007 Bias Study 
and CPS:2005 estimates of household income by race/ethnicity for children ages 3 through 6, not yet 
enrolled in kindergarten. For preschoolers of a race/ethnicity other than White, non-Hispanic, the 
percentage with a household income from $30,001 to $50,000 is higher for the CPS than the Bias Study 
sample. The rest of the estimates are comparable. 



Table 7-5. Number and percentage of children ages 3 through 6 not yet enrolled in kindergarten, 
by household income and race/ethnicity: SR-NHES:2007 Bias Study and CPS:2005 









Household income 




Race/ethnicity 


Number of 
children 


Less than 
$15,000 


$15,001 to 
$30,000 


$30,001 to 
$50,000 


More than 
$50,000 


(thousands) ' 


Percent s.e. 


Percent s.e. 


Percent s.e. 


Percent s.e. 


SR-NHES:2007 Bias 
Study 

White, non-Hispanic 


4,933 


9 2.3 


10 1.9 


21 3.8 


60 4.6 


Other 


3,801 


23 3.7 


28 5.2 


13 3.2 


36 4.9 


CPS:2005 


White, non-Hispanic 


4,882 


8 0.7 


11 0.8 


21 1.1 


60 1.3 


Other 


3,876 


24 1.3 


27 1.3 


21 1.2 


29 1.4 



# Rounds to zero. 

NOTE: Shading indicates a significant difference of 5 percentage points or more. s.e. is standard error. Current 
Population Survey percentage estimates exclude cases with missing income data. Because of rounding, percentages 
may not add to 100. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, School Readiness Survey of the 
National Household Education Surveys Program, 2007; and U.S. Department of Commerce, Bureau of the Census, 
Current Population Survey, October 2005. 
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Household income: PFI. Table 7-6 presents PFI-NHES:2007 Bias Study and CPS:2005 
estimates of the percentage of children in kindergarten through grade 12, who resided in households with 
particular income ranges. Across income categories, estimates from both surveys are comparable; the 
observed differences of 4 percentage points or less were not significant. 



Table 7-6. Percentage of children in kindergarten through grade 12, by household income: PFI- 
NHES:2007 Bias Study and CPS:2005 



Household income 


PFI-NHES:2007 Bias Study 


CPS:2005 




Percent 


S.e. 


Percent 


S.e. 


$15,000 or less 


13 


0.6 


13 




$15,001 to $30,000 


15 


1.2 


16 


HB 


$30,001 to $50,000 


17 


2.2 


20 


HB 


Over $50,000 


55 


2.2 


51 





NOTE: s.e. is standard error. Current Population Survey estimates exelude eases with missing ineome data. Beeause 
of rounding, percentages may not add to 100. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement 
in Education Survey of the National Household Education Surveys Program, 2007; and U.S. Department of 
Commerce, Bureau of the Census, Current Population Survey, October 2005. 



Household income by race/ethnicity: PFI. Table 7-7 presents PFI-NHES:2007 Bias Study 
and CPS:2005 estimates of household income by race/ethnicity for children in kindergarten through grade 
12. The estimates are comparable. 

Table 7-7. Number and percentage of children in kindergarten through grade 12, by household 
income and race/ethnicity: PFI-NHES:2007 Bias Study and CPS:2005 









Household income 




Race/ethnicity 


Number of 
children 


Less than 
$15,000 


$15,001 to 
$30,000 


$30,001 to 
$50,000 


More than 
$50,000 


(thousands) ' 


Percent s.e. 


Percent s.e. 


Percent s.e. 


Percent s.e. 


PFI-NHES:2007 Bias 
Study 

White, non-Hispanic 
Other 


30,959 

22,227 


7 0.9 

22 1.5 


9 0.8 

24 3.0 


16 3.4 

18 2.5 


69 3.6 

37 3.5 


CPS:2005 

White, non-Hispanic 
Other 


31,689 

21,639 


6 0.3 

23 0.5 


10 0.3 

24 0.5 


19 0.4 

22 0.5 


64 0.5 

32 0.6 



NOTE: s.e. is standard error. Current Population Survey percentage estimates exclude cases with missing income 
data. Because of rounding, percentages may not add to 100. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement 
in Education Survey of the National Household Education Surveys Program, 2003; and U.S. Department of 
Commerce, Bureau of the Census, Current Population Survey, October 2005. 
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Public and private schools by race/ethnicity. Estimates from PFI-NHES:2007 and 
CPS:2005 of the number and percent of children in kindergarten through grade 12 enrolled in public and 
private schools by race/ethnicity are presented in table 7-8. Estimates are comparable. 

Table 7-8. Number and percentage of children in kindergarten through grade 12 in public and 
private schools, by race/ethnicity: PFI-NHES:2007 Bias Study and CPS:2005 





PFI-NHES:2007 Bias Study 






CPS:2005 








Number of Publie 


Private 




Number of 


Publie 


Private 






ehildren 






ehildren 








Race/ethnicity 


(thousands) Pereent s.e. 


Pereent 


s.e. 


(thousands) Pereent s.e. 


Pereent 


S.e. 


White, non-Hispanic 






2.2 


31,689 


87 0.3 


13 


EB 


Other 


22,227 90 2.1 




2.1 


21,639 


94 0.3 


6 


EH 



NOTE: s.e. is standard error. Percentages include only those students for whom public/private enrollment was 
reported, that is, children whose parents indicated they were enrolled in school. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement 
in Education Survey of the National Household Education Surveys Program, 2007; and U.S. Department of 
Commerce, Bureau of the Census, Current Population Survey, October 2005. 



Family structure and parents’ highest level of education. Table 7-9 presents estimates of 
the percentage of children in kindergarten through grade 12 by family structure and by parents’ highest 
level of education for PFI-NHES:2007 Bias Study and CPS:2005-2006. The estimate for the percentage 
of children who had both mother and father in the household was 6 percentage points higher in PFI- 
NHES:2007 Bias Study (74 percent) compared to CPS:2006 (68 percent), and the percentage of children 
who had a mother only in the household was 5 percentage points lower in PFI-NHES:2007 Bias Study 
(19 percent) compared to CPS:2006 (24 percent). In addition, the percentage of children whose parents’ 
highest level of education was some college was 5 percentage points lower in PFI-NHES:2007 Bias Study 
(28 percent) compared to CPS:2005 (33 percent), and the percentage of children whose parents’ highest 
level of education was graduate school was 5 percentage points higher in PFI-NHES:2007 Bias Study (18 
percent) compared to CPS:2005 (13 percent). The reason for these differences are unclear but consistent 
with the differences observed between the CPS data and the NHES:2007 RDD survey (Hagedom et al, 
2008). 
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Table 7-9. Percentage of children in kindergarten through grade 12, by family structure and 

parents’ highest level of education: PFI-NHES:2007 Bias Study and CPS: 2005-2006 





PFI-NHES:2007 Bias Study 


CPS:2005-2006 




Family and community characteristics 


Percent 


S.e. 


Percent 


S.e. 


Family structure 


Mother and father 


74 


1.8 


68 


0.4 


Mother 


19 


1.6 


24 


0.3 


Father 


4 


0.4 


5 


0.2 


Nonparent guardian(s) 


3 


0.5 


4 


0.1 


Parents’ highest education 


Less than high school 


7 


1.2 


9 


0.2 


High school graduate 


25 


2.3 


24 


0.3 


Some college 


28 


2.1 


33 


0.4 


College graduate 


22 


2.1 


21 


0.3 


Graduate school 


18 


2.1 


13 


0.2 



NOTE: Shading indicates a significant difference of 5 percentage points or more. s.e. is standard error. Mother and 
father refer to birth, adoptive, step, or foster parents. Because of rounding, percentages may not add to 100. In 
households with two mothers/female guardians or two fathers/male guardians, parents’ highest level of education for 
PFI-NHES:2007 was derived by taking into account the education level of both parents. Current Population Survey 
percentage estimates by family structure are for children ages 5 through 17, excluding emancipated minors, from 
CPS March 2006. Current Population Survey percentage estimates by parents’ highest education are approximated 
by highest education attainment within households, from CPS October 2005. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement 
in Education Survey of the National Household Education Surveys Program (NHES), 2007; and U.S. Department of 
Commerce, Bureau of the Census, Current Population Survey, October 2005 and March 2006. 



Adult population, by sex and age. Table 7-10 shows estimates of the adult population by 
sex and age. As discussed in chapter 6, the AEWR weights were raked to control totals of age by sex from 
the CPS. Therefore, estimates from the two surveys are expected to be comparable. The age estimates for 
both males and females from AEWR-NHES:2007 Bias Study and CPS:2006 are not substantively 
different. 



no 










Table 7-10. Percentage distribution of the adult population, by sex and age: AEWR-NHES:2007 
Bias Study and CPS:2006 



Age 


AEWR-NHES:2007 Bias Study 




CPS:2006 




Male 




Female 




Male 




Female 




Estimate 


s.e. 


Estimate 


S.e. 


Estimate 


S.e. 


Estimate 


S.e. 


Total number of adults* 


















(thousands) 


104,359 


0 


112,468 


0 


104,359 


345 


112,468 


335 


1 6 to 24 years 


8 


0.9 


5 


0.7 


6 


0.1 


6 


0.1 


25 to 34 years 


8 


1.5 


10 


1.1 


9 


0.1 


9 


0.1 


35 to 44 years 


8 


1.2 


9 


0.9 


10 


0.1 


10 


0.1 


45 to 54 years 


11 


1.4 


10 


1.0 


10 


0.1 


10 


0.1 


55 years and older 


13 


0.9 


18 


0.8 


14 


0.1 


17 


0.1 



' Includes civilian, noninstitutionalized adults, ages 16 or older, not enrolled in elementary or secondary school, and 
not on active duty in the U.S. Armed Forces at the time of the interview. 

NOTE: The percentages provided in this table are cell percentages. That is, for each data set, these percentages sum 
to 100 across all age-sex cells. Due to rounding, the percentages shown here may not sum to 100. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Adult Education for Work- 
Related Reasons Survey of the National Household Education Surveys Program, 2007. U.S. Department of 
Commerce, Bureau of the Census, Current Population Survey, March 2006. 



Adult population by highest educational attainment and race/ethnicity. Race/ethnicity 
was also used in raking the AEWR weights. Since CPS:2006 is the source of the control totals for raking 
NHES:2007, estimates of number of adults in each race/ethnicity group are expected to be comparable. 
The estimates of totals for the non-Hispanic White and other race/ethnicity groups shown in table 7-11 are 
not identical, however, because the NHES:2007 Bias Study data were raked to a three-category 
race/ethnicity variable (Black, non-Hispanic; Hispanic; and White, non-Hispanic or others), whereas a 
two-category race/ethnicity variable (White, non-Hispanic versus others) is used in the comparison. 



As depicted in table 7-11, AEWR-NHES:2007 Bias Study and CPS:2006 estimates of 
educational attainment by race/ethnicity are comparable in most cases. However, the percentage of adults 
with less than a high school education for race/ethnicities other than White, non-Hispanic was 6 
percentage points lower for the AEWR-NHES:2007 Bias Study than the CPS:2006; the reason for this 
difference is unclear. 



Ill 







Table 7-11. Percentage distribution of the adult population, by highest educational attainment 
and race/ethnicity: AEWR-NHES:2007 Bias Study and CPS:2006 



Highest educational attainment 





Number of 
adults 


Less than 
high school 


High school 
diploma 


Associate’s or 
some college 


Bachelor’s or 
higher 


Race/ethnicity 


(thousands) 


Percent 


s.e. 


Percent s.e. 


Percent 


S.e. 


Percent 


S.e. 


AEWR-NHES:2007 Bias 
Study 


Total adults 


216,827 


15 


0.1 


31 1.4 


29 


1.3 


26 


1.0 


White, non-Hispanic 


153,894 


12 


1.1 


30 2.3 


29 


1.9 


29 


2.2 


All other races 


62,933 


20 


2.5 


32 4.9 


29 


3.7 


19 


3.8 


CPS:2006 


















Total adults 


216,827 


15 


0.1 


32 0.2 


28 


0.2 


26 


0.2 


White, non-Hispanic 


151,076 


10 


0.1 


32 0.2 


29 


0.2 


29 


0.2 


All other races 


65,751 


26 


0.3 


31 0.3 


24 


0.3 


19 


0.3 



' Includes civilian, noninstitutionalized adults, ages 16 or older, not enrolled in elementary or secondary school, and 
not on active duty in the U.S. Armed Forces at the time of the interview. 

NOTE: Shading indicates a significant difference of 5 percentage points or more. Detail may not sum to totals 
because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Adult Education for Work- 
Related Reasons Survey of the National Household Education Surveys Program, 2007. U.S. Department of 
Commerce, Bureau of the Census, Current Population Survey, March 2006. 

Work for pay or income in the past 12 months. In table 7-12, the estimates of 
employment status from the AEWR-NHES:2007 Bias Study and CPS:2006 are presented for adults aged 
16 or older. About 70 percent of adults reported that they worked for pay or income in the past 12 months 
in AEWR and about 69 percent reported working in CPS:2006. 



Table 7-12. Percentage of adults who worked for pay or income in the past 12 months: AEWR- 
NHES:2007 Bias Study and CPS:2006 



Work history, past 12 months 


AEWR-NHES:2007 Bias Study 


CPS:2006 




Estimate 


S.e. 


Estimate 


S.e. 


Total number of adults* (thousands) 


216,827 


t 


216,827 


t 


Worked in the past 12 months 


70 


1.8 


69 


0.2 


Did not work in the past 12 months 


30 


1.8 


31 


0.2 



t Not applicable. 

* Includes civilian, noninstitutionalized adults, ages 16 or older, not enrolled in elementary or secondary school, and 
not on active duty in the U.S. Armed Forces at the time of the interview. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Adult Education for Work- 
Related Reasons Survey of the National Household Education Surveys Program (NHES), 2007. U.S. Department of 
Commerce, Bureau of the Census, Current Population Survey, March 2006. 
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7.2.3 



Summary 



Overall, the comparisons of selected estimates from NHES:2007 Bias Study with 
comparable data from the CPS have provided an indication of the reasonableness in using the NHES:2007 
Bias Study estimates as the standard by which to evaluate bias in the RDD estimates. Although the 
estimates presented here are just some of the multitude of comparisons that could be made between 
NHES:2007 Bias Study estimates and the CPS using different variables and categorizations, this approach 
has proven useful in determining whether significant differences in estimates exist. 



7.3 Effect of the Weighting Adjustments on Bias 

The NHES:2007 estimates were produced using weights that were adjusted for nonresponse 
and calibrated (raked) to population totals. These adjustments were intended to reduce the nonresponse 
and noncoverage bias in the estimates. The process for weighting the RDD sample was similar to that for 
the Bias Study sample, as described in chapter 6. The variables used in the weighting adjustments were 
chosen because they were related to response propensity, noncoverage, and the key survey statistics. A 
complete description of the weighting process for the NHES:2007 RDD surveys can be found in 
Hagedom et al. (2008). 

In this section, the effect of the weighting adjustments on the NHES:2007 RDD sample 
estimates is evaluated. Tables 7-13, 7-14, and 7-15 provide estimates computed using weights at three 
different stages of weighting for the SR, PEI, and AEWR surveys, respectively. The first set of estimates 
uses unadjusted weights reflecting only the probabilities of selection. The second set of estimates is 
produced using weights adjusted for unit nonresponse to the Screener and the extended interview. The 
third set uses the final weights, reflecting all nonresponse and calibration adjustments. The estimates 
chosen for comparison include key outcome variables from the surveys as well as key demographics. In 
section 7.4, the same set of estimates is considered in the evaluation of bias in the published final RDD 
estimates compared to the final estimates from the Bias Study. 

As shown in tables 7-13, 7-14, and 7-15, there are no substantive differences (i.e., 
differences of 5 percentage points or more) between the unadjusted estimates and the nonresponse 
adjusted estimates. Considered together with the results provided in chapter 8, this is an indication that 
there is little evidence of nonresponse bias in the NHES:2007 estimates. 
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There are some differences between the nonresponse adjusted estimates and final estimates. 
The differences indicate potential noncoverage bias prior to raking that was reduced through the raking 
process. For the SR survey (table 7-13), estimates after the raking adjustments are lower than the 
nonresponse adjusted estimates for the proportion of preschoolers who participate in center-based care, 
recognize all colors, count to 20 or higher, and write their first name; who have parents who believe it is 
essential to prepare their child for kindergarten by teaching them the alphabet and sharing; who have a 
family member that reads to them everyday in the past week; whose parents took three or more outings 
with them in the past month; who have household incomes above $50,000; and who have both a mother 
and father in the household. In addition, the final estimates are higher than the nonresponse adjusted 
estimates for the proportion of preschoolers who live in a home that is not owned, have parents with a 
high school diploma or below, are below the poverty threshold, have household incomes below $15,000, 
and have a mother only in the household. Many of the estimates that differ before and after the raking 
adjustments are related to the variables used in raking: race/ethnicity, household income, region, 
urbanicity, home tenure, age, and enrollment status. 

For the PFI and AEWR surveys (tables 7-14 and 7-15, respectively), no substantive 
differences were found between the nonresponse adjusted estimates and final estimates for the key survey 
outcome variables. Results for the demographic variables are similar to those for the SR survey. For the 
PFI survey, the final estimates are higher than the nonresponse adjusted estimates for the proportion of 
children who live in a home that is not owned, are below the poverty threshold, and have household 
incomes below $15,000. The final estimates are lower than the nonresponse adjusted estimates for the 
proportion of children who have household incomes above $50,000 and who have both a mother and 
father in the household. Race/ethnicity, household income, region, urbanicity, home tenure, and grade 
were used in the raking adjustment for the PFI survey, and these characteristics are related to many of 
these demographic variables. For the AEWR survey, the final estimates are higher than the nonresponse 
adjusted estimates for the proportion of adults ages 24 years or younger, who do not own their home, and 
who never married. The final estimates are lower than the nonresponse adjusted estimates for the 
proportion of adults who are ages 55 years or older, who are currently married, and who have a household 
income above $50,000. Age, home tenure, and household income were among the variables used in the 
raking adjustment for the AEWR survey, and an indicator for single adults was used in the household- 
level poststratification adjustment. The differences found in this evaluation indicate the raking 
adjustments were effective in reducing noncoverage bias in these estimates. 

Differences were found for the same set of characteristics when comparing the unadjusted 
estimates to the final estimates, with a few exceptions. The difference between the unadjusted estimate 
and final estimate of the proportion of adults who have a household income above $50,000 is not of 
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substantive importance, possibly indicating nonresponse bias and noncoverage bias in the unadjusted 
estimate that were acting in opposite directions. In addition, the final estimates are lower than the 
unadjusted estimates for the proportion of preschoolers who have parents who believe it is essential to 
prepare their child for kindergarten by teaching the child numbers. The final estimates are higher than the 
unadjusted estimates for the proportion of preschoolers who are age 3 and have a household income 
between $15,001 and $30,000, the proportion of children who have parents with a high school diploma or 
below and have a mother only in the household, and the proportion of adults ages 25 to 34. These 
differences indicate potential overall bias in these unadjusted estimates that was reduced through the 
nonresponse and raking adjustments. 

The final, fully adjusted estimates from the NHES:2007 RDD sample were also compared 
with estimates from previous NHES collections, the CPS, and other relevant extant data sources, similar 
to the analysis in section 7.2. The results of these comparisons can be found in Hagedom et al (2008). The 
comparisons indicated the estimates from the NHES: 2007 RDD sample were reasonable, although some 
differences were found between the NHES:2007 and CPS estimates of household income, parents’ 
highest level of education, and family structure. 



7.4 An Examination of Overall Bias 

In addition to the comparison of estimates to external sources, an assessment of overall bias 
can be done by comparing the published estimates from the RDD survey that use the final, fully adjusted 
weights to the estimates from the Bias Study under the full treatment (completed extended interviews 
from FT/RT and FTO cells of figure 4-1; see section 5.2 for an explanation of what is meant by full and 
reduced treatment). Because the Bias Study sample and data collection effort are designed to capture both 
telephone nonrespondents and nontelephone households, differences between estimates from the Bias 
Study and estimates from the RDD sample may be a reflection of either nonresponse or noncoverage bias, 
or both. This section examines these differences in estimates of characteristics of the target populations 
for the NHES:2007 surveys. The items included in these comparisons were selected because they include 
important demographic and socioeconomic characteristics of persons and households, or because they are 
key outcome variables from the surveys. 
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Table 7-13. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the School Readiness Survey: 
NHES:2007 RDD 



SR respondents with SR respondents with SR respondents with 

unadjusted weights nonresponse adjusted weights final weights 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Participation in center-based care 


1,759 


68.2 


0.90 


68.8 


0.93 


55.3 


0.89 


Specific skills 


Recognizes all colors 


2,265 


88.4 


0.63 


88.2 


0.64 


82.5 


1.02 


Counts to 20 or higher 


1,747 


69.4 


1.09 


69.6 


1.11 


63.2 


1.30 


Recognizes all letters 


891 


36.0 


1.17 


36.6 


1.21 


31.8 


1.19 


Writes first name 


1,709 


67.9 


1.03 


67.8 


1.15 


59.8 


1.27 


Holds a pencil 


2,298 


87.6 


0.70 


87.5 


0.74 


86.8 


0.95 


Speech is often understandable to a stranger 


2,170 


84.2 


0.85 


84.2 


0.89 


81.1 


1.20 


Reads or pretends to read storybooks 


2,582 


97.9 


0.36 


97.9 


0.35 


98.0 


0.31 


Parents believe it is essential to do certain 
things to prepare child for kindergarten 


T each child the alphabet 


1,582 


62.4 


0.97 


62.2 


1.05 


56.3 


1.25 


Teach child about sharing 


1,732 


69.0 


1.10 


68.2 


1.15 


61.8 


1.37 


Teach child to read 


1,226 


47.2 


1.11 


47.6 


1.20 


45.0 


1.36 


Teach child numbers 


1,507 


59.2 


1.12 


59.0 


1.19 


54.1 


1.37 


Show child how to hold a pencil 


1,104 


43.5 


1.29 


43.7 


1.37 


40.9 


1.37 


Family member read to child everyday in the 


past week 


1,575 


62.2 


0.96 


61.7 


1.02 


55.3 


0.97 


Parents report usually doing certain reading- 
related activity with child 


Ask child what is in a picture 


1,390 


53.4 


1.13 


54.4 


1.24 


55.5 


1.26 


Stop reading and point out letters 


818 


30.6 


1.15 


31.3 


1.30 


31.1 


1.46 


Ask child to read with parent 


589 


22.1 


0.89 


22.9 


1.02 


23.6 


1.26 


Talk about the story and what happened 


1,419 


54.4 


1.24 


54.9 


1.34 


56.1 


1.44 



See notes at end of table. 









Table 7-13. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the School Readiness Survey: 
NHES:2007 RDD— Continued 







SR respondents with 
unadjusted weights 


SR respondents with 
nonresponse adjusted weights 


SR respondents with 
final weights 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate Standard 

(pereent) error 


Parents did home aetivities with ehild in the 


past week' 


1,088 


42.3 


1.20 


41.5 


1.34 


37.7 


1.20 


Parents took 3 or more outings with ehild in 


the past month^ 


1,175 


46.0 


1.12 


46.0 


1.14 


39.0 


1.10 


Child watehes 2 or more hours of TV in a 


typieal weekday 


1,538 


57.8 


1.04 


57.5 


1.04 


61.6 


1.19 


Child has a disability 


453 


17.2 


0.89 


17.4 


0.93 


17.6 


1.00 


Child’s age 


3 years 


1,098 


37.8 


1.04 


38.2 


1.14 


43.0 


1.24 


4 years 


1,159 


45.5 


1.19 


45.6 


1.31 


42.8 


1.41 


5 years and older 


376 


16.7 


0.98 


16.2 


0.99 


14.2 


0.89 


Child’s sex 


Male 


1,279 


50.1 


1.00 


50.0 


1.02 


50.3 


1.16 


Female 


1,354 


49.9 


1.00 


50.0 


1.02 


49.7 


1.16 


Household urbanieity 


Urban 


2,217 


80.8 


1.01 


81.9 


1.02 


79.9 


0.43 


Rural 


416 


19.2 


1.01 


18.1 


1.02 


20.1 


0.43 


Home tenure 


Own 


1,945 


77.4 


0.85 


75.1 


0.81 


61.3 


0.47 


Rent/other 


688 


22.6 


0.85 


24.9 


0.81 


38.7 


0.47 



See notes at end of table. 








Table 7-13. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the School Readiness Survey: 
NHES:2007 RDD— Continued 







SR respondents with 
unadjusted weights 


SR respondents with 
nonresponse adjusted weights 


SR respondents with 
final weights 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate Standard 

(pereent) error 


Parents’ edueational attainment 


High sehool diploma or below 


603 


20.2 


0.83 


20.4 


0.87 


27.8 


1.26 


Beyond high sehool diploma 


2,030 


79.8 


0.83 


79.6 


0.87 


72.2 


1.26 


Parents’ language 


Both/only parent(s) speak(s) English 


2,218 


87.7 


0.65 


86.8 


0.73 


84.9 


0.79 


One of two parents speaks English 


45 


1.4 


0.23 


1.5 


0.26 


1.5 


0.30 


No parent speaks English 


370 


11.0 


0.61 


11.7 


0.68 


13.6 


0.77 


Mothers’ employment status 


35 hours or more per week 


959 


35.6 


1.06 


35.9 


1.07 


36.4 


1.44 


Less than 35 hours per week 


597 


23.9 


0.87 


23.6 


0.90 


20.9 


0.94 


Looking for work 


108 


3.6 


0.36 


3.5 


0.37 


5.5 


0.70 


Not in labor foree 


927 


35.5 


1.08 


35.5 


1.16 


35.8 


1.48 


No mother in household 


42 


1.5 


0.28 


1.4 


0.28 


1.5 


0.30 


Poverty status 


Poor 


412 


13.3 


0.66 


13.5 


0.71 


22.5 


0.81 


Nonpoor 


2,221 


86.7 


0.66 


86.5 


0.71 


77.5 


0.81 


Household ineome 


Less than $15,000 


251 


7.7 


0.52 


7.9 


0.55 


14.5 


0.93 


$15,001 to $30,000 


356 


12.1 


0.84 


12.5 


0.90 


17.4 


1.09 


$30,001 to $50,000 


446 


16.6 


0.88 


16.5 


0.85 


17.1 


0.82 


More than $50,000 


1,580 


63.6 


1.03 


63.0 


1.01 


51.0 


0.82 



See notes at end of table. 









Table 7-13. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the School Readiness Survey: 
NHES:2007 RDD— Continued 



Characteristic 




SR respondents with 
unadjusted weights 


SR respondents with 
nonresponse adjusted weights 


SR respondents with 
final weights 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Family structure 


Mother and father 


2,192 


85.6 


0.73 


85.3 


0.79 


78.7 


0.96 


Mother 


346 


11.2 


0.63 


11.7 


0.70 


17.2 


0.92 


Father 


40 


1.4 


0.25 


1.3 


0.23 


1.4 


0.28 


Nonparent guardian(s) 


55 


1.7 


0.27 


1.7 


0.30 


2.6 


0.54 



' Told child a story; taught child letters, words, or numbers; taught child songs or music; did arts and crafts with child; played sports, active games or 
exercised together; and played board games or did puzzles with child. 



^ Any three or more of the following: Visited a library; visited a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or 
historical site; visited a zoo or aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting 
event (outside of school) in which the child was not a player. 

NOTE: Shading indicates a significant difference of 5 percentage points or more between the unadjusted and final estimate. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, School Readiness (SR) Survey of the National Household Education 
Surveys Program, 2007. 




Table 7-14. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the Parent and Family 
Involvement Survey: NHES:2007 RDD 







PFI respondents with 
unadjusted weights 


PFI respondents with 
nonresponse adjusted weights 


PFI respondents with 
final weights 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate Standard 

(pereent) error 


Parents partieipate in 5 or more aetivities in 
the student’s sehool' 


5,576 


55.1 


0.59 


55.4 


0.60 


53.0 


0.62 


Parents report sehool provides information 
very well 

About how student is doing in sehool 


6,385 


61.7 


0.60 


61.7 


0.65 


60.9 


0.76 


About how to help student with his/her 
homework 


4,740 


45.6 


0.57 


46.2 


0.59 


46.6 


0.65 


About why student is plaeed in partieular 
groups or elasses 


4,481 


43.7 


0.55 


44.0 


0.58 


44.5 


0.65 


About how to help student plan for eollege 
or voeational sehool 


2,064 


33.4 


0.74 


33.7 


0.79 


34.0 


0.86 


About the family’s expeeted role at 
student’s sehool 


5,024 


48.4 


0.60 


48.9 


0.64 


48.4 


0.72 


Parent reports being very satisfied with 4 
or more aspeets of the student’s sehool^ 


7,263 


70.3 


0.54 


70.3 


0.56 


69.8 


0.62 


Parents partieipated in 5 or more home 
learning aetivities^ 


2,213 


46.4 


0.87 


46.7 


0.94 


47.0 


1.02 


Parents took 3 or more outings with student 
in the past month"* 


5,321 


51.1 


0.67 


51.5 


0.72 


49.5 


0.77 


Parents eheek to see that student’s homework 
gets done 


8,190 


83.7 


0.49 


84.3 


0.53 


85.4 


0.46 


Parents reeeived information about free 
tutoring 


4,552 


42.5 


0.58 


42.9 


0.62 


43.9 


0.65 



See notes at end of table. 







Table 7-14. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the Parent and Family 
Involvement Survey: NHES:2007 RDD — Continued 







PFI respondents with 
unadjusted weights 


PFI respondents with 
nonresponse adjusted weights 


PFI respondents with 
final weights 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate Standard 

(pereent) error 


Parent expeets student to earn a eollege degree 


or higher 


4,360 


72.4 


0.87 


72.4 


0.93 


69.6 


1.02 


Family plans to help pay for student’s 


edueation after high sehool 


4,700 


83.6 


0.68 


83.3 


0.73 


81.3 


0.84 


Student partieipated in sehool aetivities 


5,965 


59.0 


0.66 


58.0 


0.72 


56.0 


0.76 


Student has a disability 


2,463 


23.2 


0.58 


22.8 


0.61 


23.9 


0.68 


Student’s sex 


Male 


5,498 


51.8 


0.60 


51.6 


0.64 


51.8 


0.74 


Female 


5,183 


48.2 


0.60 


48.4 


0.64 


48.2 


0.74 


Home tenure 


Own 


8,438 


81.3 


0.49 


78.9 


0.46 


70.0 


0.24 


Rent/other 


2,243 


18.7 


0.49 


21.1 


0.46 


30.0 


0.24 


Parents’ edueational attainment 


High sehool diploma or below 


2,578 


22.7 


0.53 


23.3 


0.52 


27.8 


0.56 


Beyond high sehool diploma 


8,103 


77.3 


0.53 


76.7 


0.52 


72.2 


0.56 


Parents’ language 


Both/only parent(s) speak(s) English 


9,437 


90.5 


0.31 


89.6 


0.33 


88.5 


0.34 


One of two parents speaks English 


159 


1.2 


0.12 


1.3 


0.13 


1.4 


0.16 


No parent speaks English 


1,085 


8.3 


0.30 


9.1 


0.30 


10.2 


0.32 



See notes at end of table. 









Table 7-14. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the Parent and Family 
Involvement Survey: NHES:2007 RDD — Continued 



K) 

K) 







PFI respondents with 
unadjusted weights 


PFI respondents with 
nonresponse adjusted weights 


PFI respondents with 
final weights 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate Standard 

(pereent) error 


Mothers’ employment status 


35 hours or more per week 


4,993 


44.7 


0.60 


45.2 


0.67 


44.2 


0.66 


Less than 35 hours per week 


2,290 


23.9 


0.47 


23.4 


0.48 


21.8 


0.44 


Looking for work 


393 


3.4 


0.19 


3.4 


0.21 


4.6 


0.33 


Not in labor foree 


2,611 


24.9 


0.53 


24.8 


0.57 


26.1 


0.59 


No mother in household 


394 


3.2 


0.18 


3.2 


0.19 


3.3 


0.21 


Poverty status 


Poor 


1,291 


11.7 


0.37 


12.4 


0.42 


19.2 


0.33 


Nonpoor 


9,390 


88.3 


0.37 


87.6 


0.42 


80.8 


0.33 


Household ineome 


Less than $15,000 


824 


6.6 


0.32 


6.9 


0.35 


12.2 


0.32 


$15,001 to $30,000 


1,321 


11.4 


0.38 


12.1 


0.43 


15.0 


0.40 


$30,001 to $50,000 


1,799 


16.3 


0.46 


16.6 


0.47 


16.4 


0.45 


More than $50,000 


6,737 


65.7 


0.54 


64.4 


0.56 


56.4 


0.46 


Family stmeture 


Mother and father 


7,995 


79.1 


0.45 


78.0 


0.50 


72.7 


0.54 


Mother 


1,876 


14.8 


0.40 


15.8 


0.44 


20.0 


0.55 


Father 


356 


2.9 


0.18 


2.9 


0.19 


3.0 


0.21 


Nonparent guardian(s) 


454 


3.3 


0.21 


3.3 


0.25 


4.3 


0.39 


Sehool type 


Publie 


8,978 


86.9 


0.40 


86.8 


0.45 


88.2 


0.45 


Private 


1,392 


13.1 


0.40 


13.2 


0.45 


11.8 


0.45 



See notes at end of table. 










Table 7-14. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the Parent and Family 
Involvement Survey: NHES:2007 RDD — Continued 



Characteristic 




PFI respondents with 
unadjusted weights 


PFI respondents with 
nonresponse adjusted weights 


PFI respondents with 
final weights 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


School size 
















Under 300 


1,480 


15.5 


0.47 


14.9 


0.51 


14.9 


0.57 


300-599 


3,142 


31.3 


0.62 


30.9 


0.62 


31.3 


0.69 


600-999 


2,756 


27.0 


0.57 


27.4 


0.58 


26.7 


0.60 


1,000 or more 


2,910 


26.1 


0.56 


26.7 


0.56 


27.0 


0.52 



' Any 5 or more of the following: Attended a general school meeting; attended a meeting of the parent-teacher organization or association; went to a 
regularly scheduled parent-teacher conference with the student’s teacher; attended a school or class event because of the student; served as a volunteer in 
the student’s classroom or elsewhere in the school; participated in fundraising for the school; served on a school committee; and met with a guidance 
counselor in person. 

^ Any 4 or more of the following: School student attends this year; teachers student has this year; academic standards of the school; order and discipline at 
the school; and way that school staff interact with parents. 

^ Any 5 or more of the following: Told student a story; did arts and crafts with student; played sports, active games or exercised together; worked on 
projects such as building, making, or fixing something with student not as a chore; talked with student about family history or ethnic heritage; and played 
board games or did puzzles with student. 

Any 3 or more of the following: Visited a library; visited a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or 
historical site; visited a zoo or aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting 
event (outside of school) in which the child was not a player. 

NOTE: Shading indicates a significant difference of 5 percentage points or more between the unadjusted and final estimate. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement in Education (PFI) Survey of the 
National Household Education Surveys Program, 2007. 







Table 7-15. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the Adult Education for 
Work-Related Reasons Survey: NHES:2007 RDD 



AEWR respondents with AEWR respondents with AEWR respondents with 

unadjusted weights nonresponse adjusted weights final weights 



Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Partieipates in adult edueation for work 
related reasons 


3,356 


36.0 


0.68 


37.0 


0.73 


37.6 


0.95 


Partieipates in employer-supported AEWR 


2,379 


74.2 


0.91 


75.2 


1.14 


73.2 


1.58 


Partieipates in distanee edueation 


1,964 


56.2 


1.21 


56.5 


1.33 


54.4 


1.67 


Partieipates in program to earn a eollege 
or university degree 


694 


7.0 


0.31 


7.6 


0.38 


9.7 


0.55 


Partieipates in program to earn a voeational 
or teehnieal diploma 


383 


3.7 


0.25 


3.7 


0.31 


4.1 


0.38 


Partieipates in formal apprentieeship 
program 


79 


0.9 


0.12 


1.0 


0.14 


1.4 


0.25 


Partieipates in work-related training or 
eourses 


2,899 


31.5 


0.70 


32.2 


0.73 


31.2 


0.90 


Partieipates in 4 or more informal work 
related learning aetivities* 


2,334 


27.5 


0.68 


28.6 


0.83 


27.6 


1.07 


Has any eondition that limits ability to 
work 


1,190 


14.1 


0.47 


12.9 


0.50 


13.5 


0.68 



See notes at end of table. 







Table 7-15. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the Adult Education for Work- 
Related Reasons Survey: NHES:2007 RDD — Continued 



AEWR respondents with AEWR respondents with AEWR respondents with 

unadjusted weights nonresponse adjusted weights final weights 



Charaeteristie 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Age 


1 6 to 24 years 


438 


5.8 


0.33 


6.7 


0.41 


12.1 


0.70 


25 to 34 years 


738 


10.4 


0.61 


12.0 


0.77 


16.4 


0.92 


35 to 44 years 


1,091 


17.8 


0.66 


19.9 


0.94 


19.4 


0.81 


45 to 54 years 


1,595 


22.8 


0.76 


23.0 


0.83 


20.6 


0.83 


55 years and older 


3,848 


43.2 


0.79 


38.4 


0.90 


31.6 


0.41 


Home tenure 


Own 


6,289 


83.7 


0.62 


83.4 


0.73 


70.3 


0.34 


Rent/other 


1,421 


16.3 


0.62 


16.6 


0.73 


29.7 


0.34 


Marital status 


Never married 


1,320 


13.5 


0.49 


14.0 


0.55 


22.3 


0.80 


Currently married 


4,329 


68.1 


0.69 


70.4 


0.75 


61.5 


0.86 


Other 


2,061 


18.4 


0.58 


15.6 


0.58 


16.2 


0.58 


Household ineome 


Less than $15,000 


838 


9.4 


0.42 


8.7 


0.47 


11.1 


0.35 


$15,001 to $30,000 


1,182 


14.4 


0.56 


13.0 


0.63 


15.0 


0.57 


$30,001 to $50,000 


1,495 


18.7 


0.65 


18.5 


0.70 


20.9 


0.92 


More than $50,000 


4,195 


57.5 


0.80 


59.8 


0.90 


53.0 


0.90 


Language spoken most at home 


English 


7,277 


93.7 


0.45 


92.6 


0.58 


90.1 


0.63 


Spanish 


244 


3.6 


0.35 


4.3 


0.45 


6.0 


0.48 


Other language 


136 


2.0 


0.27 


2.2 


0.31 


2.3 


0.30 


English and Spanish equally 


52 


0.7 


0.12 


0.9 


0.23 


1.6 


0.63 


English and other language equally 


t 


t 




t 


t 


t 


t 



See notes at end of table. 








Table 7-15. Unadjusted, nonresponse adjusted, and final estimates for various characteristics from the Adult Education for Work- 
Related Reasons Survey: NHES:2007 RDD — Continued 



Characteristic 




AEWR respondents with 
unadjusted weights 


AEWR respondents with 
nonresponse adjusted weights 


AEWR respondents with 
final weights 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Employment status 
















Employed 


4,311 


56.3 


0.85 


58.4 


1.00 


58.2 


1.09 


Unemployed but looking for 
















work 


250 


3.5 


0.41 


3.7 


0.50 


5.7 


0.76 


Not in the labor force 


3,149 


40.2 


0.79 


37.9 


0.95 


36.1 


1.04 



X Reporting standards not met. 

' Any 4 or more of the following: Received on-the-job demonstrations of equipment, techniques, or procedures by a supervisor or coworker; received 
other supervised training or mentoring on the job; self-paced study using books, procedures manuals, audio tapes, or videos; self-paced study using 
computer-based software tutorials; attended “brown-bag” or informal presentations; attended conferences, trade shows, or conventions related to the 
adult’s work or profession; and read professional journals, trade publications, or work-related magazines. 

NOTE: Shading indicates a significant difference of 5 percentage points or more between the unadjusted and final estimate. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Adult Education for Work-Related Reasons Survey of the National 
Household Education Surveys Program, 2007. 







Table 7-16 contains estimates from the RDD sample and the Bias Study sample of various 
characteristics from the SR Survey. The Bias Study sample yielded larger estimates of the percentage of 
preschoolers who can count to 20 or higher and the percentage of preschoolers whose speech is often 
understandable to a stranger. Additionally, the Bias Study estimate of the percentage of preschoolers who 
watch two or more hours of television in a typical weekday is higher. Although there are some differences 
in estimates between the two samples, it is likely that these differences were found mainly as a result of 
having examined so many characteristics."'^ There is no systematic relationship among these differences 
that would be indicative of bias. 

There is a difference in the estimates of the percentage of preschoolers whose mothers are 
not in the labor force (36 percent from the RDD sample versus 26 percent from the Bias Study sample). 
This difference may be an indication of accessibility, with mothers who are not in the labor force being 
more available and more willing to complete the interview by telephone than mothers with other 
employment status. One other curious difference is in the sex distribution of preschoolers; the Bias Study 
sample estimated 62 percent of preschoolers to be male, compared to 50 percent for the RDD sample and 
52 percent for the CPS sample in October 2005; the reason for this difference is unclear."'^ 

In table 7-17, estimates from the RDD sample and the Bias Study sample of characteristics 
from the PFI Survey are compared. There are no significant differences of 5 percentage points or more. 
However, there are some smaller differences of statistical significance. For instance, the estimated 
percentage of parents who report the school provides information very well about how to help the student 
plan for college or vocational school is 34 percent for the RDD sample, which is 13 percent lower than 
the Bias Study estimate. In addition, the estimated percentage of students in schools of size 1,000 or 
more is 12 percent lower for the RDD sample than the Bias Study sample (27 percent versus 3 1 percent). 



Another possible explanation for the differences is the later data collection for the field portion of the Bias Study sample. However, the mean 
and median age-in-months at the time of interview of preschoolers in the RDD sample were compared to that for the Bias Study sample, and no 
significant differences were found. While the Bias Study preschoolers may have had more months of preschool/daycare, this can not be tested, 
and therefore there is not evidence to attribute the differences to the data collection schedules. 

The skewed sex distribution for the Bias Study sample is also evident before weighting adjustments (see Appendix G) and for the reduced 
effort (see chapter 8). To evaluate the effect of the skewed sex distribution on the analysis of overall bias, the Bias Study weights were re-raked, 
including sex as an additional raking dimension. The analysis in Table 7-16 was then reproduced with the new weights. The conclusions 
remained the same, with a few exceptions. First, the difference between the RDD sample and Bias Study sample estimates of the percentage of 
preschoolers who can count to 20 or higher was no longer of substantive importance after re-raking. The estimated bias was -5.2 percentage 
points before re-raking and -4.8 percentage points after. Second, the difference in the percentage of preschoolers of age 4 years was statistically 
significant after re-raking. The estimated bias was 5.5 percentage points (not significant) before re-raking and 6.7 percentage points after. 
Finally, the difference in the percentage of preschoolers whose mother was looking for work increased from -4.5 percentage points (not of 
substantive importance) before re-raking to a statistically significant and substantive difference of -5.5 percentage points after. 
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Table 7-16. Estimates of overall bias for various characteristics from the School Readiness Survey: NHES:2007 RDD and NHES:2007 
Bias Study 





All RDD sample SR respondents 


All Bias Study sample SR 
respondents 


Bias' 




Relative 

bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Participation in center-based care 


1,759 


55.3 


0.89 


177 


51.1 


2.29 


4.2 


2.46 


8.2 


Specific skills 


Recognizes all colors 


2,265 


82.5 


1.02 


249 


85.6 


2.34 


-3.1 


2.55 


-3.6 


Counts to 20 or higher 


1,747 


63.2 


1.30 


198 


68.4 


2.27 


-5.2 


2.61 


-7.6 


Recognizes all letters 


891 


31.8 


1.19 


93 


29.4 


2.89 


2.4 


3.12 


8.2 


Writes first name 


1,709 


59.8 


1.27 


193 


60.4 


3.08 


-0.6 


3.33 


-0.9 


Holds a pencil 


2,298 


86.8 


0.95 


252 


86.7 


1.90 


# 


2.12 


0.1 


Speech is often understandable 
to a stranger 


2,170 


81.1 


1.20 


249 


87.3 


2.17 


-6.2 


2.48 


-7.1 


Reads or pretends to read 
storybooks 


2,582 


98.0 


0.31 


289 


99.2 


0.45 


-1.2 


0.55 


-1.2 


Parents believe it is essential to do 
certain things to prepare child 
for kindergarten 
Teach child the alphabet 


1,582 


56.3 


1.25 


171 


57.7 


2.72 


-1.4 


2.99 


-2.5 


Teach child about sharing 


1,732 


61.8 


1.37 


177 


57.8 


2.97 


4.0 


3.27 


6.9 


Teach child to read 


1,226 


45.0 


1.36 


139 


48.0 


3.47 


-3.0 


3.73 


-6.3 


Teach child numbers 


1,507 


54.1 


1.37 


155 


51.7 


2.64 


2.4 


2.98 


4.6 


Show child how to hold a pencil 


1,104 


40.9 


1.37 


121 


37.9 


2.88 


3.0 


3.19 


8.0 


Family member read to child 


everyday in the past week 


1,575 


55.3 


0.97 


169 


56.9 


4.02 


-1.6 


4.14 


-2.8 



See notes at end of table. 







Table 7-16. Estimates of overall bias for various characteristics from the School Readiness Survey: NHES:2007 RDD and NHES:2007 
Bias Study — Continued 





All RDD sample SR respondents 


All Bias Study sample SR 
respondents 


Bias' 




Relative 

bias^ 


Characteristie 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Parents report usually doing eertain 
reading-related aetivity with 
ehild 

Ask child what is in a picture 


1,390 


55.5 


1.26 


159 


52.9 


4.00 


2.6 


4.20 


4.9 


Stop reading and point out 
letters 


818 


31.1 


1.46 


95 


31.9 


4.14 


-0.8 


4.39 


-2.6 


Ask child to read with parent 


589 


23.6 


1.26 


84 


29.7 


3.05 


-6.1 


3.30 


-20.6 


Talk about the story and what 
happened 


1,419 


56.1 


1.44 


165 


60.3 


3.18 


-4.3 


3.49 


-7.0 


Parents did home activities with 
child in the past week^ 


1,088 


37.7 


1.20 


123 


41.7 


3.23 


-4.0 


3.45 


-9.5 


Parents took 3 or more outings 
with child in the past month"^ 


1,175 


39.0 


1.10 


138 


46.0 


3.71 


-7.0 


3.87 


-15.3 


Child watches 2 or more hours of 
TV in a typical weekday 


1,538 


61.6 


1.19 


179 


71.1 


2.10 


-9.5 


2.41 


-13.3 


Child has a disability 


453 


17.6 


1.00 


37 


14.1 


3.34 


3.5 


3.49 


24.8 


Child’s age 
3 years 


1,098 


43.0 


1.24 


128 


43.9 


2.86 


-0.9 


3.12 


-2.1 


4 years 


1,159 


42.8 


1.41 


116 


37.3 


2.64 


5.5 


2.99 


14.8 


5 years and older 


376 


14.2 


0.89 


48 


18.8 


3.08 


-4.6 


3.20 


-24.4 



See notes at end of table. 






Table 7-16. Estimates of overall bias for various characteristics from the School Readiness Survey: NHES:2007 RDD and NHES:2007 
Bias Study — Continued 



All Bias Study sample SR 
All RDD sample SR respondents respondents 



Bias' 



Relative 

bias^ 



Sample Estimate Standard Sample Estimate Standard Estimate Standard Estimate 



Characteristie size (pereent) error size (pereent) error (percent) error (percent) 



Child’s sex 


Male 


1,279 


50.3 


1.16 


169 


62.3 


2.97 


-12.0 


3.19 


-19.3 


Female 


1,354 


49.7 


1.16 


123 


37.7 


2.97 


12.0 


3.19 


31.9 


Household urbanicity 




















Urban 


2,217 


79.9 


0.43 


247 


80.8 


1.62 


-0.9 


1.67 


-1.2 


Rural 


416 


20.1 


0.43 


45 


19.2 


1.62 


0.9 


1.67 


4.9 


Home tenure 




















Own 


1,945 


61.3 


0.47 


201 


62.1 


0.87 


-0.8 


0.99 


-1.3 


Rent/other 


688 


38.7 


0.47 


91 


37.9 


0.87 


0.8 


0.99 


2.1 


Parents’ educational attainment 




















High school diploma or below 


603 


27.8 


1.26 


91 


34.5 


3.80 


-6.7 


4.00 


-19.5 


Beyond high school diploma 


2,030 


72.2 


1.26 


201 


65.5 


3.80 


6.7 


4.00 


10.3 


Parents’ language 




















Both/only parent(s) speak(s) 
English 

One of two parents speaks 


2,218 


84.9 


0.79 


254 


88.0 


2.48 


-3.1 


2.60 


-3.5 


English 


45 


1.5 


0.30 








0.5 


0.78 


50.0 


No parent speaks English 


370 


13.6 


0.77 


29 


11.0 


2.60 


2.6 


2.71 


23.9 



See notes at end of table. 




Table 7-16. Estimates of overall bias for various characteristics from the School Readiness Survey: NHES:2007 RDD and NHES:2007 
Bias Study — Continued 





All RDD sample SR respondents 


All Bias Study sample SR 
respondents 


Bias' 




Relative 

bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Mothers’ employment status 


35 hours or more per week 


959 


36.4 


1.44 


111 


41.6 


3.34 


-5.2 


3.64 


-12.6 


Less than 35 hours per week 


597 


20.9 


0.94 


69 


20.4 


3.21 


0.5 


3.35 


2.3 


Looking for work 


108 


5.5 


0.70 


23 


10.0 


2.49 


-4.5 


2.59 


-44.7 


Not in labor foree 


927 


35.8 


1.48 


83 


26.1 


2.65 


9.7 


3.04 


37.2 


No mother in household 


42 


1.5 


0.30 


6! 


1.9! 


1.18! 


-0.4 


1.21 


-20.2 


Poverty status 


Poor 


412 


22.5 


0.81 


80 


24.5 


2.19 


-2.0 


2.34 


-8.1 


Nonpoor 


2,221 


77.5 


0.81 


212 


75.5 


2.19 


2.0 


2.34 


2.6 


Household ineome 


Less than $15,000 


251 


14.5 


0.93 


52 


15.3 


1.71 


-0.8 


1.94 


-5.3 


$15,001 to $30,000 


356 


17.4 


1.09 


62 


17.6 


2.39 


-0.2 


2.63 


-0.9 


$30,001 to $50,000 


446 


17.1 


0.82 


50 


17.6 


2.77 


-0.5 


2.89 


-2.9 


More than $50,000 


1,580 


51.0 


0.82 


128 


49.5 


2.92 


1.5 


3.03 


3.0 



See notes at end of table. 








Table 7-16. Estimates of overall bias for various characteristics from the School Readiness Survey: NHES:2007 RDD and NHES:2007 
Bias Study — Continued 





All RDD sample SR respondents 


All Bias Study sample SR 
respondents 


Bias' 




Relative 

bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Family structure 
Mother and father 


2,192 


78.7 


0.96 


219 


73.4 


4.36 


5.3 


4.46 


7.2 


Mother 


346 


17.2 


0.92 


60 


22.4 


3.09 


-5.2 


3.23 


-23.1 


Father 


40 


1.4 


0.28 


6! 


1.9! 


1.18! 


-0.5 


1.21 


-25.5 


Nonparent guardian(s) 


55 


2.6 


0.54 


7 


2.3 


0.96 


0.3 


1.10 


11.6 



# Rounds to zero. 

! Interpret with eaution. 

X Reporting standards not met. 

' Bias is estimated as the differenee between the “all RDD sample SR respondents” estimate and the “all Bias Study sample SR respondents” estimate. 

^ Relative bias is estimated as the bias estimate divided by the “all Bias Study sample SR respondents” estimate, and is expressed as a pereentage. 

^ Told ehild a story; taught ehild letters, words, or numbers; taught ehild songs or musie; did arts and erafts with ehild; played sports, aetive games or exereised 
together; and played board games or did puzzles with child. 

^ Any three or more of the following: Visited a library; visited a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or 
historical site; visited a zoo or aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting event 
(outside of school) in which the child was not a player. 

NOTE: Shading indicates a significant difference of 5 percentage points or more. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, School Readiness (SR) Survey of the National Household Education Surveys 
Program, 2007. 



Table 7-17. Estimates of overall bias for various characteristics from the Parent and Family Involvement Survey: NHES:2007 RDD and 
NHES:2007 Bias Study 





All RDD sample PFI respondents 


All Bias Study sample PFI 
respondents 


Bias' 




Relative 

bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Parents partieipate in 5 or more 
aetivities in the student’s 
sehool^ 


5,576 


53.0 


0.62 


566 


52.1 


2.22 


0.9 


2.30 


1.7 


Parents report sehool provides 
information very well 
About how student is doing in 
sehool 


6,385 


60.9 


0.76 


668 


58.2 


2.20 


2.7 


2.33 


4.6 


About how to help student 
with his/her homework 


4,740 


46.6 


0.65 


525 


47.9 


2.07 


-1.3 


2.17 


-2.7 


About why student is plaeed 
in partieular groups or 
elasses 


4,481 


44.5 


0.65 


496 


45.6 


1.67 


-1.1 


1.79 


-2.4 


About how to help student 
plan for eollege or 
voeational sehool 


2,064 


34.0 


0.86 


237 


38.9 


1.65 


-4.9 


1.86 


-12.6 


About the family’s expeeted 
role at student’s sehool 


5,024 


48.4 


0.72 


539 


46.9 


1.93 


1.5 


2.06 


3.2 


Parent reports being very satisfied 
with 4 or more aspeets of the 
student’s sehool"* 


7,263 


69.8 


0.62 


798 


73.7 


1.87 


-3.9 


1.97 


-5.3 


Parents partieipated in 5 or more 
home learning aetivities^ 


2,213 


47.0 


1.02 


229 


42.2 


2.28 


4.8 


2.50 


11.4 



See notes at end of table. 



Table 7-17. Estimates of overall bias for various characteristics from the Parent and Family Involvement Survey: NHES:2007 RDD and 
NHES:2007 Bias Study — Continued 





All RDD sample PFI respondents 


All Bias Study sample PFI 
respondents 


Bias' 




Relative 

bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Parents took 3 or more outings 
with student in the past month^ 


5,321 


49.5 


0.77 


559 


50.3 


2.09 


-0.8 


2.23 


-1.6 


Parents check to see that student’s 
homework gets done 


8,190 


85.4 


0.46 


885 


87.9 


0.99 


-2.5 


1.09 


-2.8 


Parents received information about 
free tutoring 


4,552 


43.9 


0.65 


465 


42.6 


2.15 


1.3 


2.25 


3.1 


Parent expects student to earn a 
college degree or higher 


4,360 


69.6 


1.02 


424 


67.2 


2.87 


2.4 


3.05 


3.6 


Family plans to help pay for 
student’s education after high 
school 


4,700 


81.3 


0.84 


475 


83.1 


2.19 


-1.8 


2.35 


-2.2 


Student participated in school 
activities 


5,965 


56.0 


0.76 


624 


55.8 


1.89 


0.2 


2.04 


0.4 


Student has a disability 


2,463 


23.9 


0.68 


257 


23.0 


2.16 


0.9 


2.26 


3.9 


Student’s sex 
Male 


5,498 


51.8 


0.74 


575 


51.8 


2.10 


# 


2.23 


# 


Female 


5,183 


48.2 


0.74 


548 


48.2 


2.10 


# 


2.23 


# 


Home tenure 
Own 


8,438 


70.0 


0.24 


837 


69.7 


0.80 


0.3 


0.84 


0.4 


Rent/other 


2,243 


30.0 


0.24 


286 


30.3 


0.80 


-0.3 


0.84 


-1.0 



See notes at end of table. 



Table 7-17. Estimates of overall bias for various characteristics from the Parent and Family Involvement Survey: NHES:2007 RDD and 
NHES:2007 Bias Study — Continued 



All Bias Study sample PFI Relative 





All RDD sample PFI respondents 




respondents 




Bias' 




bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Parents’ educational attainment 


High school diploma or below 


2,578 


27.8 


0.56 


326 


31.7 


2.31 


-3.9 


2.38 


-12.3 


Beyond high school diploma 


8,103 


72.2 


0.56 


797 


68.3 


2.31 


3.9 


2.38 


5.7 


Parents’ language 

Both/only parent(s) speak(s) 


English 


9,437 


88.5 


0.34 


1012 


90.9 


1.29 


-2.4 


1.33 


-2.6 


One of two parents speaks 


English 


159 


1.4 


0.16 


22 


2.1 


0.61 


-0.7 


0.63 


-33.3 


No parent speaks English 


1,085 


10.2 


0.32 


66 


7.0 


1.17 


3.2 


1.21 


45.7 


Mothers’ employment status 


35 hours or more per week 


4,993 


44.2 


0.66 


521 


47.0 


2.12 


-2.8 


2.22 


-6.0 


Less than 35 hours per week 


2,290 


21.8 


0.44 


260 


21.1 


1.68 


0.7 


1.74 


3.3 


Looking for work 


393 


4.6 


0.33 


45 


4.1 


0.56 


0.5 


0.65 


12.2 


Not in labor force 


2,611 


26.1 


0.59 


241 


23.4 


1.24 


2.7 


1.37 


11.5 


No mother in household 


394 


3.3 


0.21 


56 


4.4 


0.55 


-1.1 


0.59 


-25.0 


Poverty status 


Poor 


1,291 


19.2 


0.33 


221 


19.7 


1.05 


-0.5 


1.10 


-2.5 


Nonpoor 


9,390 


80.8 


0.33 


902 


80.3 


1.05 


0.5 


1.10 


0.6 


Household income 


Less than $15,000 


824 


12.2 


0.32 


153 


12.9 


0.64 


-0.7 


0.72 


-5.4 


$15,001 to $30,000 


1,321 


15.0 


0.40 


179 


15.2 


1.22 


-0.2 


1.28 


-1.3 


$30,001 to $50,000 


1,799 


16.4 


0.45 


171 


16.6 


2.23 


-0.2 


2.27 


-1.2 


More than $50,000 


6,737 


56.4 


0.46 


620 


55.3 


2.22 


1.1 


2.27 


2.0 



See notes at end of table. 



Table 7-17. Estimates of overall bias for various characteristics from the Parent and Family Involvement Survey: NHES:2007 RDD and 
NHES:2007 Bias Study — Continued 





All RDD sample PFI respondents 


All Bias Study sample PFI 
respondents 


Bias' 




Relative 

bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Family structure 


Mother and father 


7,995 


72.7 


0.54 


823 


74.1 


1.88 


-1.4 


1.96 


-1.9 


Mother 


1,876 


20.0 


0.55 


204 


18.6 


1.61 


1.4 


1.70 


7.5 


Father 


356 


3.0 


0.21 


54 


4.0 


0.42 


-1.0 


0.47 


-25.0 


Nonparent guardian(s) 


454 


4.3 


0.39 


42 


3.3 


0.55 


1.0 


0.67 


30.3 


School type 


Public 


8,978 


88.2 


0.45 


989 


89.9 


1.65 


-1.7 


1.71 


-1.9 


Private 


1,392 


11.8 


0.45 


114 


10.1 


1.65 


1.7 


1.71 


16.8 


School size 


Under 300 


1,480 


14.9 


0.57 


167 


14.6 


1.51 


0.3 


1.61 


2.1 


300-599 


3,142 


31.3 


0.69 


321 


28.8 


2.59 


2.5 


2.68 


8.7 


600-999 


2,756 


26.7 


0.60 


286 


25.8 


1.81 


0.9 


1.91 


3.5 


1 ,000 or more 


2,910 


27.0 


0.52 


323 


30.8 


1.65 


-3.8 


1.73 


-12.3 



# Rounds to zero. 

* Bias is estimated as the difference between the “all RDD sample PFI respondents” estimate and the “all Bias Study sample PFI respondents” estimate. 

^ Relative bias is estimated as the bias estimate divided by the “all Bias Study sample PFI respondents” estimate, and is expressed as a percentage. 

^ Any 5 or more of the following: Attended a general school meeting; attended a meeting of the parent-teacher organization or association; went to a regularly 
scheduled parent-teacher conference with the student’s teacher; attended a school or class event because of the student; served as a volunteer in the student’s 
classroom or elsewhere in the school; participated in fundraising for the school; served on a school committee; and met with a guidance counselor in person. 

Any 4 or more of the following: School student attends this year; teachers student has this year; academic standards of the school; order and discipline at the 
school; and way that school staff interact with parents. 

^ Any 5 or more of the following: Told student a story; did arts and crafts with student; played sports, active games or exercised together; worked on projects 
such as building, making, or fixing something with student not as a chore; talked with student about family history or ethnic heritage; and played board games or 
did puzzles with student. 

® Any 3 or more of the following: Visited a library; visited a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or historical 
site; visited a zoo or aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting event (outside of 
school) in which the child was not a player. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement in Education (PFI) Survey of the National 
Household Education Surveys Program, 2007. 



Table 7-18 contains a comparison of estimates from the RDD and Bias Study samples of 
characteristics from the AEWR Survey. There are differences in the marital status estimates, with the 
RDD sample yielding a higher estimate (62 percent) of currently married adults than the Bias Study 
sample (56 percent). With an estimate of 56 percent from the March 2006 Current Population Survey 
(CPS),"'"' it is likely that the bias is in the RDD estimate, and that this may be indicative of the relative 
inaccessibility and undercoverage of unmarried adults. 



““ Based on independent tabulations of the March 2006 CPS data. 
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Table 7-18. Estimates of overall bias for various characteristics from the Adult Education for Work-Related Reasons Survey: 
NHES:2007 RDD and NHES:2007 Bias Study 





All RDD sample AEWR 
respondents 


All Bias Study sample AEWR 
respondents 


Bias' 




Relative 

bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Partieipates in adult edueation for 
work-related reasons 


3,356 


37.6 


0.95 


495 


39.1 


1.76 


-1.5 


2.00 


-3.8 


Partieipates in employer-supported 
AEWR 


2,379 


73.2 


1.58 


347 


70.0 


3.57 


3.2 


3.90 


4.6 


Partieipates in distanee edueation 


1,964 


54.4 


1.67 


285 


60.1 


2.98 


-5.7 


3.41 


-9.4 


Partieipates in program to earn a 
eollege or university degree 


694 


9.7 


0.55 


124 


9.4 


0.81 


0.3 


0.98 


3.0 


Partieipates in program to earn a 
voeational or teehnieal diploma 


383 


4.1 


0.38 


56 


4.2 


0.70 


-0.1 


0.80 


-1.9 


Partieipates in formal 

apprentieeship program 


79 


1.4 


0.25 


5! 


0.5! 


0.33! 


0.9 


0.42 


197.9 


Partieipates in work-related 
training or eourses 


2,899 


31.2 


0.90 


416 


33.2 


1.49 


-2.0 


1.74 


-6.0 


Partieipates in 4 or more informal 
work-related learning 
aetivities^ 


2,334 


27.6 


1.07 


324 


29.2 


2.25 


-1.6 


2.49 


-5.4 


Has any eondition that limits 
ability to work 


1,190 


13.5 


0.68 


153 


15.2 


2.63 


-1.7 


2.71 


-11.2 



See notes at end of table. 





Table 7-18. Estimates of overall bias for various characteristics from the Adult Education for Work-Related Reasons Survey: 
NHES:2007 RDD and NHES:2007 Bias Study— Continued 





All RDD sample AEWR 
respondents 


All Bias Study sample AEWR 
respondents 


Bias' 




Relative 

bias^ 


Charaeteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Age 


1 6 to 24 years 


438 


12.1 


0.70 


104 


13.1 


1.17 


-1.0 


1.36 


-7.8 


25 to 34 years 


738 


16.4 


0.92 


168 


17.5 


1.86 


-1.1 


2.08 


-6.3 


35 to 44 years 


1,091 


19.4 


0.81 


131 


16.9 


1.31 


2.5 


1.54 


14.8 


45 to 54 years 


1,595 


20.6 


0.83 


217 


21.4 


1.75 


-0.8 


1.94 


-3.6 


55 years and older 


3,848 


31.6 


0.41 


445 


31.1 


1.20 


0.5 


1.27 


1.6 


Home tenure 


Own 


6,289 


70.3 


0.34 


794 


70.1 


0.79 


0.2 


0.86 


0.3 


Rent/other 


1,421 


29.7 


0.34 


271 


29.9 


0.79 


-0.2 


0.86 


-0.8 


Marital status 


Never married 


1,320 


22.3 


0.80 


249 


24.2 


1.42 


-1.9 


1.63 


-7.7 


Currently married 


4,329 


61.5 


0.86 


531 


55.9 


2.06 


5.6 


2.23 


10.1 


Other 


2,061 


16.2 


0.58 


285 


20.0 


1.69 


-3.8 


1.79 


-18.9 


Household income 


Less than $15,000 


838 


11.1 


0.35 


194 


10.6 


0.54 


0.5 


0.64 


4.5 


$15,001 to $30,000 


1,182 


15.0 


0.57 


212 


16.1 


1.24 


-1.1 


1.36 


-6.7 


$30,001 to $50,000 


1,495 


20.9 


0.92 


188 


23.1 


1.59 


-2.2 


1.84 


-9.3 


More than $50,000 


4,195 


53.0 


0.90 


471 


50.3 


1.30 


2.8 


1.58 


5.5 



See notes at end of table. 




Table 7-18. Estimates of overall bias for various characteristics from the Adult Education for Work-Related Reasons Survey: 
NHES:2007 RDD and NHES:2007 Bias Study— Continued 





All RDD sample AEWR 
respondents 


All Bias Study sample AEWR 
respondents 


Bias' 




Relative 

bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Language spoken most at home 


English 


7,277 


90.1 


0.63 


999 


92.4 


1.60 


-2.3 


1.72 


-2.5 


Spanish 


244 


6.0 


0.48 


39 


4.7 


1.08 


1.3 


1.19 


26.3 


Other language 


136 


2.3 


0.30 


18 


2.1 


0.89 


0.2 


0.94 


9.0 


English and Spanish equally 


52 


1.6 


0.63 


8 


0.7 


0.31 


0.9 


0.70 


142.4 


English and other language 
equally 




















Employment status 


Employed 


4,311 


58.2 


1.09 


627 


61.2 


1.93 


-3.0 


2.21 


-4.8 


Unemployed but looking for 
work 


250 


5.7 


0.76 


44 


5.0 


1.28 


0.7 


1.49 


13.1 


Not in the labor force 


3,149 


36.1 


1.04 


394 


33.8 


1.35 


2.3 


1.70 


6.8 



! Interpret with caution. 

X Reporting standards not met. 

' Bias is estimated as the difference between the “all RDD sample AEWR respondents” estimate and the “all Bias Study sample AEWR respondents” 
estimate. 

^ Relative bias is estimated as the bias estimate divided by the “all Bias Study sample AEWR respondents” estimate, and is expressed as a percentage. 

^ Any 4 or more of the following: Received on-the-job demonstrations of equipment, techniques, or procedures by a supervisor or coworker; received 
other supervised training or mentoring on the job; self-paced study using books, procedures manuals, audio tapes, or videos; self-paced study using 
computer-based software tutorials; attended “brown-bag” or informal presentations; attended conferences, trade shows, or conventions related to the 
adult’s work or profession; and read professional journals, trade publications, or work-related magazines. 

NOTE: Shading indicates a significant difference of 5 percentage points or more. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Adult Education for Work-Related Reasons Survey of the National 
Household Education Surveys Program, 2007. 



8. COMPONENTS OF BIAS 



Chapter 7 discussed an examination of overall bias in estimates from the NHES:2007 
surveys. In this chapter, we examine separately the two major potential components of bias, nonresponse 
bias and noncoverage bias. As discussed in chapter 1, nonresponse bias occurs when sampled units fail to 
respond to the survey request and those units differ in some systematic fashion from those that do 
respond. Noncoverage bias occurs when units that are not included on the sampling frame differ in a 
systematic way from units that are included on the sampling frame. Section 8.1 examines nonresponse 
bias in estimates from the NHES:2007 surveys, and section 8.2 contains an evaluation of noncoverage 
biasinNHES:2007. 



8.1 An Examination of Nonresponse Bias 

As discussed in chapter 3, prior to selecting the sample for the Bias Study, telephone 
numbers were matched to addresses to the extent possible. When a telephone number was available for a 
sampled address, the case was attempted by the Telephone Research Center (TRC) using the standard 
telephone data collection protocol. When the Screener could not be completed by telephone (due to 
noncontact, unit nonresponse, or the lack of a matching telephone number), the case was sent to the field 
for attempts to complete an interview, as described in chapter 4. 

Thus, nonresponse bias can be assessed by examining differences in estimates from the 
reduced effort (completed extended interviews from FT/RT cells of figure 4-1) and the frill effort 
(completed extended interviews from FT/RT and FTO cells of figure 4-1)."^^ Nonmatched cases (Bias 
Study cases with no telephone number match available) could not be attempted by the TRC, but were 
attempted in the field. Thus, it should be noted that the reduced effort estimates were calculated using 
data from both the TRC respondents and the nonmatched cases, in order to eliminate the effect 
nonmatched cases would have on estimates of nonresponse bias. The weighting and estimation approach 
used to include the nonmatched cases in the reduced effort estimates is discussed in chapter 6. 



As discussed in chapter 5, in the reduced effort, the Telephone Research Center (TRC) status is treated as the final household status for Bias 
Study cases attempted in the TRC. In the full effort, the final household status is based on efforts in both the TRC and in the field. For example, a 
case that was a final refusal in the TRC but responded to the Screener in the field is treated as a nonrespondent in the reduced effort but as a 
respondent in the full effort. A total of 4,894 cases was respondents with the full effort, and 4,235 were respondents under the reduced effort. 
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To better understand the rationale for including the nonmatched cases in both sets, consider 
the difference between the reduced and full effort estimates as 



-yp 

= H treaty i + Y.nm^iyi ~ ^Utrc^iy^ +Z field^’tyi + 



where the subscript sum over tre is the respondents in the TRC, nm sums over the respondents in the 
nonmatched cases, field sums over the respondents in the field, and the case weights ( w,- and w- ) are 

defined in chapter 6 for the two effort levels. If the weights were identical for the two sets, then it is clear 
that including the nonmatched cases would have no effect on the estimated difference. The weights do 
differ, mainly due to the adjustments to population control totals, so this simple result does not hold 
exactly but the inclusion of the nonmatched cases in both sets should still essentially cancel. 

The analyses in Chapter 7 established the effectiveness of the nonresponse adjustments and 
calibration adjustments in reducing the amount of potential bias in the examined variables. Thus, this 
chapter focuses on measuring the potential bias in fully weighted estimates (i.e., those computed using 
nonresponse adjustments and raking). This provides the basis for evaluating the potential for bias due to 
nonresponse and noncoverage in the published estimates. Nonetheless, as was demonstrated in section 
7.3, analyses looking at base-weighted estimates are informative and all analyses shown in this section are 
replicated using base weights in appendix G. Note that the unadjusted base weight is the product of the 
household base weight and the reciprocal of the child’s or adult’s within- household selection probability. 

Tables 8-1, 8-2, and 8-3 contain estimates of nonresponse bias for various characteristics 
from the School Readiness (SR), Parent and Family Involvement in Education (PFI), and Adult Education 
for Work-Related Reasons (AEWR) surveys, respectively. As is the case with all tables in this chapter, 
the estimates reflect skip patterns; the denominators of the percentages include only respondents to the 
item. As these tables demonstrate, the comparison of the full effort estimates to the reduced effort 
estimates yields no indication of nonresponse bias. There are no significant differences of 5 percentage 
points or more among any of the comparisons made in the tables. Comparisons using unadjusted base 
weights give the same conclusions. 
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Table 8-1. Estimates of nonresponse bias for various characteristics from the School Readiness Survey: NHES:2007 Bias Study 





Reduced treatment SR respondents 


Full treatment SR respondents 


Bias' 




Relative bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Participation in center-based care 


143 


51.0 


2.13 


177 


51.1 


2.29 


-0.1 


0.85 


-0.2 


Specific skills 


Recognizes all colors 


207 


85.0 


2.35 


249 


85.6 


2.34 


-0.6 


0.47 


-0.7 


Counts to 20 or higher 


165 


68.5 


2.11 


198 


68.4 


2.27 


0.2 


1.03 


0.3 


Recognizes all letters 


78 


29.6 


3.59 


93 


29.4 


2.89 


0.2 


1.11 


0.7 


Writes first name 


158 


59.4 


3.13 


193 


60.4 


3.08 


-0.9 


1.07 


-1.5 


Holds a pencil 


210 


86.6 


2.13 


252 


86.7 


1.90 


-0.1 


0.66 


-0.1 


Speech is often understandable 
to a stranger 


207 


86.2 


2.43 


249 


87.3 


2.17 


-1.1 


0.45 


-1.3 


Reads or pretends to read 
storybooks 


242 


98.8 


0.62 


289 


99.2 


0.45 


-0.3 


0.19 


-0.3 


Parents believe it is essential to do 
certain things to prepare child 
for kindergarten 
Teach child the alphabet 


143 


57.7 


2.30 


171 


57.7 


2.72 


# 


0.96 


# 


Teach child about sharing 


149 


58.7 


2.50 


177 


57.8 


2.97 


0.9 


1.10 


1.6 


Teach child to read 


117 


47.2 


4.06 


139 


48.0 


3.47 


-0.8 


1.06 


-1.7 


Teach child numbers 


129 


51.5 


2.47 


155 


51.7 


2.64 


-0.2 


0.85 


-0.4 


Show child how to hold a pencil 


99 


37.2 


3.11 


121 


37.9 


2.88 


-0.7 


0.98 


-1.8 


Family member read to child 


everyday in the past week 


145 


57.4 


4.43 


169 


56.9 


4.02 


0.5 


1.35 


0.9 



See notes at end of table. 






Table 8-1. Estimates of nonresponse bias for various characteristics from the School Readiness Survey: NHES:2007 Bias Study — 
Continued 



Reduced treatment SR respondents Full treatment SR respondents Bias* Relative bias^ 





Sample 


Estimate 


Standard 


Sample 


Estimate 


Standard 


Estimate 


Standard 


Estimate 


Characteristic 


size 


(percent) 


error 


size 


(percent) 


error 


(percent) 


error 


(percent) 


Parents report usually doing certain 
reading-related activity with 
child 




















Ask child what is in a picture 
Stop reading and point out 


133 


52.8 


3.86 


159 


52.9 


4.00 


-0.1 


0.82 


-0.2 


letters 


81 


31.6 


4.17 


95 


31.9 


4.14 


-0.4 


0.76 


-1.3 


Ask child to read with parent 
Talk about the story and what 


72 


29.4 


3.35 


84 


29.7 


3.05 


-0.3 


0.90 


-1.0 


happened 


140 


59.6 


3.29 


165 


60.3 


3.18 


-0.7 


0.91 


-1.2 


Parents did home activities with 




















child in the past week^ 


104 


41.1 


3.14 


123 


41.7 


3.23 


-0.6 


1.01 


-1.4 


Parents took 3 or more outings with 




















child in the past month"* 


115 


46.3 


3.82 


138 


46.0 


3.71 


0.3 


1.29 


0.7 


Child watches 2 or more hours of 




















TV in a typical weekday 


151 


71.4 


2.30 


179 


71.1 


2.10 


0.4 


0.96 


0.6 


Child has a disability 


30 


14.8 


3.63 


37 


14.1 


3.34 


0.7 


0.70 


5.0 


Child’s age 




















3 years 


108 


44.6 


2.84 


128 


43.9 


2.86 


0.7 


1.19 


1.6 


4 years 


95 


35.6 


2.30 


116 


37.3 


2.64 


-1.7 


1.21 


-4.6 


5 years and older 


42 


19.8 


3.19 


48 


18.8 


3.08 


1.0 


0.85 


5.3 


Child’s sex 




















Male 


145 


63.0 


3.42 


169 


62.3 


2.97 


0.7 


1.11 


1.1 


Female 


100 


37.0 


3.42 


123 


37.7 


2.97 


-0.7 


1.11 


-1.9 



See notes at end of table. 



Table 8-1. Estimates of nonresponse bias for various characteristics from the School Readiness Survey: NHES:2007 Bias Study — 
Continued 



Reduced treatment SR respondents Full treatment SR respondents Bias* Relative bias^ 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Household urbanicity 


Urban 


205 


80.1 


1.97 


247 


80.8 


1.62 


-0.7 


0.44 


-0.9 


Rural 


40 


19.9 


1.97 


45 


19.2 


1.62 


0.7 


0.44 


3.6 


Home tenure 


Own 


165 


62.2 


0.72 


201 


62.1 


0.87 


0.1 


0.36 


0.2 


Rent/other 


80 


37.8 


0.72 


91 


37.9 


0.87 


-0.1 


0.36 


-0.3 


Parents’ educational attainment 


High school diploma or below 


76 


34.3 


3.40 


91 


34.5 


3.80 


-0.3 


0.83 


-0.9 


Beyond high school diploma 


169 


65.7 


3.40 


201 


65.5 


3.80 


0.3 


0.83 


0.5 


Parents’ language 

Both/only parent(s) speak(s) 


English 


212 


87.7 


2.53 


254 


88.0 


2.48 


-0.3 


0.31 


-0.3 


One of two parents speaks 


English 


t 


t 


t 




t 


t 


t 


t 


t 


No parent speaks English 


26 


11.3 


2.62 


29 


11.0 


2.60 


0.3 


0.32 


2.7 


Mothers’ employment status 


35 hours or more per week 


89 


39.2 


3.09 


111 


41.6 


3.34 


-2.5 


0.66 


-6.0 


Less than 35 hours per week 


55 


20.2 


3.24 


69 


20.4 


3.21 


-0.2 


0.71 


-1.0 


Looking for work 


23 


11.3 


2.58 


23 


10.0 


2.49 


1.3 


0.31 


13.0 


Not in labor force 


74 


27.8 


2.85 


83 


26.1 


2.65 


1.7 


0.59 


6.5 


No mother in household 


4! 


1.5! 


1.15! 


6! 


1.9! 


1.18! 


-0.4 


0.48 


-21.1! 



See notes at end of table. 





Table 8-1. Estimates of nonresponse bias for various characteristics from the School Readiness Survey: NHES:2007 Bias Study — 
Continued 





Reduced treatment SR respondents 


Full treatment SR respondents 


Bias' 




Relative bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Poverty status 


Poor 


61 


24.7 


2.29 


80 


24.5 


2.19 


0.2 


0.39 


0.8 


Nonpoor 


184 


75.3 


2.29 


212 


75.5 


2.19 


-0.2 


0.39 


-0.3 


Household income 


Less than $15,000 


39 


15.1 


1.95 


52 


15.3 


1.71 


-0.2 


0.50 


-1.3 


$15,001 to $30,000 


47 


17.5 


2.65 


62 


17.6 


2.39 


-0.1 


0.54 


-0.6 


$30,001 to $50,000 


42 


16.6 


2.71 


50 


17.6 


2.77 


-1.0 


0.64 


-5.7 


More than $50,000 


117 


50.8 


3.23 


128 


49.5 


2.92 


1.3 


0.92 


2.6 


Family structure 


Mother and father 


187 


74.3 


4.61 


219 


73.4 


4.36 


0.9 


0.96 


1.2 


Mother 


49 


22.3 


3.39 


60 


22.4 


3.09 


-0.1 


0.88 


-0.4 


Father 


4! 


1.5! 


1.15! 


6! 


1.9! 


1.18! 


-0.4 


0.48 


-21.1! 


Nonparent guardian(s) 


5 


1.9 


0.83 


7 


2.3 


0.96 


-0.4 


0.46 


-17.4 



# Rounds to zero. 

! Interpret with caution. 

J Reporting standards not met. 

* Bias is estimated as the difference between the reduced treatment SR respondent estimate and the full treatment SR respondent estimate. 

^ Relative bias is estimated as the bias estimate divided by the full treatment SR respondent estimate, and is expressed as a percentage. 

^ Told child a story; taught child letters, words, or numbers; taught child songs or music; did arts and crafts with child; played sports, active games or exercised 
together; and played board games or did puzzles with child. 

Any three or more of the following: Visited a library; visited a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or historical 
site; visited a zoo or aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting event (outside of 
school) in which the child was not a player. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, School Readiness (SR) Survey of the National Household Education Surveys 
Program, 2007. 




Table 8-2. Estimates of nonresponse bias for various characteristics from the Parent and Family Involvement in Education (PEI) 
Survey: NHES:2007 Bias Study 





Redueed treatment PFI respondents 


Full treatment PFI respondents 


Bias* 




Relative bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Parents partieipate in 5 or more 
aetivities in the student’s 
sehool^ 


478 


51.0 


2.39 


566 


52.1 


2.22 


-1.1 


0.53 


-2.1 


Parents report sehool provides 
information very well 
About how student is doing in 
sehool 


568 


58.5 


2.15 


668 


58.2 


2.20 


0.3 


0.49 


0.5 


About how to help student 
with his/her homework 


449 


48.6 


2.04 


525 


47.9 


2.07 


0.7 


0.50 


1.5 


About why student is plaeed 
in partieular groups or 
elasses 


419 


45.9 


1.57 


496 


45.6 


1.67 


0.3 


0.54 


0.7 


About how to help student 
plan for eollege or 
voeational sehool 


205 


40.1 


1.60 


237 


38.9 


1.65 


1.1 


0.72 


2.8 


About the family’s expeeted 
role at student’s sehool 


460 


47.5 


1.92 


539 


46.9 


1.93 


0.6 


0.54 


1.3 


Parent reports being very satisfied 
with 4 or more aspeets of the 
student’s sehool"* 


685 


74.4 


1.80 


798 


73.7 


1.87 


0.8 


0.69 


1.1 


Parents partieipated in 5 or more 
home learning aetivities^ 


194 


42.0 


2.57 


229 


42.2 


2.28 


-0.3 


0.62 


-0.7 


Parents took 3 or more outings with 
student in the past month'’ 


472 


49.5 


2.25 


559 


50.3 


2.09 


-0.8 


0.47 


-1.6 



See notes at end of table. 



Table 8-2. Estimates of nonresponse bias for various characteristics from the Parent and Family Involvement in Education (PEI) 
Survey: NHES:2007 Bias Study — Continued 





Redueed treatment PFI respondents 


Full treatment PFI respondents 


Bias' 




Relative bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Parents eheek to see that student’s 
homework gets done 


748 


87.3 


1.06 


885 


87.9 


0.99 


-0.5 


0.37 


-0.6 


Parents reeeived information about 
free tutoring 


396 


42.1 


2.25 


465 


42.6 


2.15 


-0.4 


0.66 


-0.9 


Parent expeets student to earn a 
eollege degree or higher 


369 


67.8 


3.38 


424 


67.2 


2.87 


0.6 


0.87 


0.9 


Family plans to help pay for 
student’s edueation after high 
sehool 


414 


83.5 


2.24 


475 


83.1 


2.19 


0.4 


0.74 


0.5 


Student partieipated in sehool 
aetivities 


531 


55.8 


2.25 


624 


55.8 


1.89 


0.1 


0.72 


0.2 


Student has a disability 


211 


21.7 


2.15 


257 


23.0 


2.16 


-1.3 


0.63 


-5.7 


Student’s sex 
Male 


500 


52.1 


2.19 


575 


51.8 


2.10 


0.3 


0.49 


0.6 


Female 


459 


47.9 


2.19 


548 


48.2 


2.10 


-0.3 


0.49 


-0.6 


Flousehold urbanieity 
Urban 


797 


81.7 


0.08 


933 


81.8 


0.06 


# 


0.03 


# 


Rural 


162 


18.3 


0.08 


190 


18.2 


0.06 


# 


0.03 


# 


Flome tenure 
Own 


708 


69.7 


0.77 


837 


69.7 


0.80 


# 


0.13 


# 


Rent/other 


251 


30.3 


0.77 


286 


30.3 


0.80 


# 


0.13 


# 



See notes at end of table. 



Table 8-2. Estimates of nonresponse bias for various characteristics from the Parent and Family Involvement in Education (PEI) 
Survey: NHES:2007 Bias Study — Continued 



Reduced treatment PFI respondents Full treatment PFI respondents Bias* Relative bias^ 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Parents’ educational attainment 


Fligh school diploma or below 


269 


31.0 


2.35 


326 


31.7 


2.31 


-0.7 


0.54 


-2.2 


Beyond high school diploma 


690 


69.0 


2.35 


797 


68.3 


2.31 


0.7 


0.54 


1.0 


Parents’ language 

Both/only parent(s) speak(s) 


English 


861 


90.8 


1.18 


1012 


90.9 


1.29 


-0.1 


0.24 


-0.1 


One of two parents speaks 


English 


20 


2.1 


0.55 


22 


2.1 


0.61 


0.0 


0.15 


0.0 


No parent speaks English 


59 


7.1 


1.18 


66 


7.0 


1.17 


0.1 


0.15 


1.4 


Mothers’ employment status 


35 hours or more per week 


445 


46.4 


2.27 


521 


47.0 


2.12 


-0.6 


0.58 


-1.3 


Less than 35 hours per week 


219 


21.3 


2.01 


260 


21.1 


1.68 


0.2 


0.42 


0.9 


Looking for work 


41 


4.3 


0.68 


45 


4.1 


0.56 


0.2 


0.28 


4.9 


Not in labor force 


207 


23.7 


1.43 


241 


23.4 


1.24 


0.3 


0.74 


1.3 


No mother in household 


47 


4.3 


0.51 


56 


4.4 


0.55 


-0.1 


0.24 


-2.3 


Poverty status 


Poor 


164 


19.2 


1.04 


221 


19.7 


1.05 


-0.5 


0.24 


-2.5 


Nonpoor 


795 


80.8 


1.04 


902 


80.3 


1.05 


0.5 


0.24 


0.6 


Flousehold income 


Less than $15,000 


115 


13.1 


0.64 


153 


12.9 


0.64 


0.1 


0.33 


0.8 


$15,001 to $30,000 


141 


14.9 


1.33 


179 


15.2 


1.22 


-0.3 


0.37 


-2.0 


$30,001 to $50,000 


152 


16.4 


2.26 


171 


16.6 


2.23 


-0.2 


0.39 


-1.2 


More than $50,000 


551 


55.7 


2.32 


620 


55.3 


2.22 


0.4 


0.48 


0.7 



See notes at end of table. 



Table 8-2. Estimates of nonresponse bias for various characteristics from the Parent and Family Involvement in Education (PEI) 
Survey: NHES:2007 Bias Study — Continued 





Reduced treatment PFI respondents 


Full treatment PFI respondents 


Bias' 




Relative bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Family structure 


Mother and father 


706 


73.2 


2.04 


823 


74.1 


1.88 


-0.9 


0.69 


-1.2 


Mother 


172 


19.6 


1.83 


204 


18.6 


1.61 


0.9 


0.69 


4.8 


Father 


45 


3.9 


0.36 


54 


4.0 


0.42 


-0.2 


0.24 


-5.0 


Nonparent guardian(s) 


36 


3.3 


0.59 


42 


3.3 


0.55 


0.1 


0.11 


3.0 


School type 


Public 


840 


89.3 


1.71 


989 


89.9 


1.65 


-0.6 


0.36 


-0.7 


Private 


101 


10.7 


1.71 


114 


10.1 


1.65 


0.6 


0.36 


5.9 


School size 


Under 300 


148 


15.2 


1.50 


167 


14.6 


1.51 


0.6 


0.33 


4.1 


300-599 


275 


28.3 


2.63 


321 


28.8 


2.59 


-0.5 


0.43 


-1.7 


600-999 


241 


26.1 


1.84 


286 


25.8 


1.81 


0.4 


0.70 


1.6 


1,000 or more 


272 


30.3 


1.88 


323 


30.8 


1.65 


-0.5 


0.58 


-1.6 



# Rounds to zero. 

* Bias is estimated as the differenee between the redueed treatment PFI respondent estimate and the full treatment PFI respondent estimate. 

^ Relative bias is estimated as the bias estimate divided by the full treatment PFI respondent estimate, and is expressed as a pereentage. 

^ Any five or more of the following: Attended a general sehool meeting; attended a meeting of the parent-teaeher organization or assoeiation; went to a regularly 
seheduled parent-teaeher eonferenee with the student’s teaeher; attended a sehool or elass event beeause of the student; served as a volunteer in the student’s 
elassroom or elsewhere in the sehool; partieipated in fundraising for the sehool; served on a sehool eommittee; and met with a guidanee eounselor in person. 

Any four or more of the following: Sehool student attends this year; teachers student has this year; academic standards of the school; order and discipline at the 
school; and way that school staff interact with parents. 

^ Any five or more of the following: Told student a story; did arts and crafts with student; played sports, active games or exercised together; worked on projects 
such as building, making, or fixing something with student not as a chore; talked with student about family history or ethnic heritage; and played board games or 
did puzzles with student. 

® Any three or more of the following: Visited a library; visited a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or historical 
site; visited a zoo or aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting event (outside of 
school) in which the student was not a player. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement in Education (PFI) Survey of the National 
Flousehold Education Surveys Program, 2007. 




Table 8-3. Estimates of nonresponse bias for various characteristics from the Adult Education for Work-Related Reasons (AEWR) 
Survey: NHES:2007 Bias Study 





Redueed treatment AEWR 
respondents 


Full treatment AEWR respondents 


Bias' 




Relative bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Partieipates in adult edueation for 
work-related reasons 


433 


39.3 


2.02 


495 


39.1 


1.76 


0.2 


0.63 


0.5 


Partieipates in employer-supported 
AEWR 


307 


69.5 


3.68 


347 


70.0 


3.57 


-0.4 


0.42 


-0.6 


Partieipates in distanee edueation 


251 


61.3 


2.96 


285 


60.1 


2.98 


1.2 


0.86 


2.0 


Partieipates in program to earn a 
eollege or university degree 


108 


9.4 


0.90 


124 


9.4 


0.81 


# 


0.22 


# 


Partieipates in program to earn a 
voeational or teehnieal diploma 


48 


4.1 


0.69 


56 


4.2 


0.70 


-0.1 


0.18 


-2.4 


Partieipates in formal 

apprentieeship program 


5! 


0.5! 


0.32! 


5! 


0.5! 


0.33! 


# 


0.03 


# 


Partieipates in work-related 
training or eourses 


368 


33.8 


1.66 


416 


33.2 


1.49 


0.6 


0.62 


1.8 


Partieipates in 4 or more informal 
work-related learning aetivities^ 


290 


29.8 


2.49 


324 


29.2 


2.25 


0.7 


0.58 


2.4 


Has any eondition that limits ability 
to work 


139 


15.3 


2.61 


153 


15.2 


2.63 


0.1 


0.33 


0.7 



See notes at end of table. 



Table 8-3. Estimates of nonresponse bias for various characteristics from the Adult Education for Work-Related Reasons (AEWR) 
Survey: NHES:2007 Bias Study — Continued 





Redueed treatment AEWR 
respondents 


Full treatment AEWR respondents 


Bias' 




Relative bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Age 


1 6 to 24 years 


89 


13.6 


1.20 


104 


13.1 


1.17 


0.5 


0.28 


3.8 


25 to 34 years 


143 


16.8 


1.93 


168 


17.5 


1.86 


-0.7 


0.38 


-4.0 


35 to 44 years 


111 


17.2 


1.42 


131 


16.9 


1.31 


0.3 


0.42 


1.8 


45 to 54 years 


195 


21.3 


1.76 


217 


21.4 


1.75 


-0.1 


0.38 


-0.5 


55 years and older 


393 


31.2 


1.16 


445 


31.1 


1.20 


0.1 


0.19 


0.3 


Census region 


Northeast 


233 


25.6 


4.64 


259 


25.7 


4.68 


-0.1 


0.29 


-0.4 


Midwest 


207 


21.8 


3.08 


235 


21.3 


2.89 


0.4 


0.27 


1.9 


South 


260 


32.4 


2.95 


314 


32.8 


3.28 


-0.4 


0.39 


-1.2 


West 


231 


20.2 


0.00 


257 


20.2 


0.00 


# 


0.00 


# 


Home tenure 


Own 


693 


70.6 


0.70 


794 


70.1 


0.79 


0.5 


0.24 


0.7 


Rent/other 


238 


29.4 


0.70 


271 


29.9 


0.79 


-0.5 


0.24 


-1.7 


Household size 


1 person 


245 


9.4 


0.67 


278 


9.6 


0.65 


-0.2 


0.34 


-2.1 


More than 1 person 


686 


90.6 


0.67 


787 


90.4 


0.65 


0.2 


0.34 


0.2 


Marital status 


Never married 


217 


24.5 


1.43 


249 


24.2 


1.42 


0.3 


0.48 


1.2 


Currently married 


468 


56.5 


2.00 


531 


55.9 


2.06 


0.6 


0.66 


1.1 


Other 


246 


19.0 


1.77 


285 


20.0 


1.69 


-1.0 


0.45 


-5.0 



See notes at end of table. 



Table 8-3. Estimates of nonresponse bias for various characteristics from the Adult Education for Work-Related Reasons (AEWR) 
Survey: NHES:2007 Bias Study — Continued 



Reduced treatment AEWR 

respondents Full treatment AEWR respondents Bias* Relative bias^ 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Household income 


Less than $15,000 


154 


10.5 


0.62 


194 


10.6 


0.54 


-0.1 


0.17 


-0.9 


$15,001 to $30,000 


173 


16.0 


1.33 


212 


16.1 


1.24 


-0.1 


0.23 


-0.6 


$30,001 to $50,000 


171 


22.2 


1.56 


188 


23.1 


1.59 


-0.9 


0.49 


-3.9 


More than $50,000 


433 


51.4 


1.36 


471 


50.3 


1.30 


1.1 


0.53 


2.2 


Language spoken most at home 


English 


876 


93.1 


1.44 


999 


92.4 


1.60 


0.7 


0.29 


0.8 


Spanish 


33 


4.5 


1.00 


39 


4.7 


1.08 


-0.3 


0.17 


-6.4 


Other language 


15 


1.8 


0.81 


18 


2.1 


0.89 


-0.3 


0.18 


-14.3 


English and Spanish equally 


7 


0.6 


0.31 


8 


0.7 


0.31 


# 


0.03 


# 


English and other language 


equally 


# 


# 


# 


t 


t 




t 


t 


t 


Employment status 


Employed 


551 


61.2 


2.04 


627 


61.2 


1.93 


# 


0.50 


# 


Unemployed but looking for 


work 


36 


4.6 


1.46 


44 


5.0 


1.28 


-0.4 


0.54 


-8.0 


Not in the labor force 


344 


34.2 


1.47 


394 


33.8 


1.35 


0.4 


0.56 


1.2 



# Rounds to zero. 

! Interpret with caution. 

J Reporting standards not met. 

* Bias is estimated as the difference between the reduced treatment AEWR respondent estimate and the full treatment AEWR respondent estimate. 

^ Relative bias is estimated as the bias estimate divided by the full treatment AEWR respondent estimate, and is expressed as a percentage. 

^ Any four or more of the following: Received on-the-job demonstrations of equipment, techniques, or procedures by a supervisor or coworker; received other 
supervised training or mentoring on the job; self-paced study using books, procedures manuals, audio tapes, or videos; self-paced study using computer-based 
software tutorials; attended “brown-bag” or informal presentations; attended conferences, trade shows, or conventions related to the adult’s work or profession; 
and read professional journals, trade publications, or work-related magazines. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Adult Education for Work-Related Reasons Survey of the National Household 
Education Surveys Program, 2007. 



It should be noted that because these results are based on estimates from the full effort (with 
overall unit response rates of 54.0 percent, 51.4 percent, and 41.0 percent for the School Readiness (SR), 
Parent and Family Involvement in Education (PFI), and Adult Education for Work-Related Reasons 
(AEWR) surveys, respectively), this examination of nonresponse bias can only account for bias that 
would be evident as unit response is increased through the field effort; this comparison does not account 
for bias due to the proportion that failed to respond to the Screener or the extended interview survey even 
after the field effort. The results shown here give no indication that those who respond to the field effort 
but not to the telephone data collection effort are different from those who respond to the telephone 
attempts. 



Section 4.2 included discussions of two data collection instruments that were used in the 
field effort but are not part of the standard data collection effort: the Interviewer Observation Form (lOF) 
and the Maximum Call postcard. Field interviewers were instructed to complete the lOF on the first visit 
to the household, before approaching the household, and to not change any responses based on 
information gleaned later. For the 5,122 cases sent to the field for in-person attempts,"^® a total of 4,600 
lOFs were returned. As noted in section 4.2, of the 635 maximum call Screener cases and maximum call 
postcards distributed, 222 maximum call postcards, or 35 percent, were returned. These instruments 
provide data that can be used to extend the nonresponse bias analysis beyond the respondents who were 
interviewed in the field. Tables 8-4, 8-5, and 8-6 provide the results from the analysis of this remaining 
nonresponse. Since the tables include estimates for field nonrespondents, this analysis was done using 
base weighted estimates rather than fully weighted estimates. 

Table 8-4 contains the frequency distributions of the lOF items; separate distributions are 
given for field respondents and for field nonrespondents to evaluate the potential bias due to not obtaining 
a 100 percent response rate in the field. The field respondents and field nonrespondents differ in 
socioeconomic classification, evidence of children, and the appearance of signs for private security 
services. Interviewers classified the household as working class or poor for 40 percent of the field 
respondents compared to 34 percent of the field nonrespondents. The percentage of addresses for which 
the field interviewer indicated evidence of children was higher for field respondents than for field 
nonrespondents (23 percent versus 12 percent). Also, the percentage of cases for which the field 
interviewer indicated finding no houses/apartments on the block with signs for private security services 
was higher for field respondents than for field nonrespondents (64 percent versus 54 percent). 



* This includes the 18 Bias Study cases completed during TRC efforts that were verified as having a correct address-telephone matched in the 
field. 



154 




Table 8-4. Distributions of characteristics collected in the Interviewer Observation Form (lOF), 
for field respondents and field nonrespondents separately: NHES:2007 Bias Study 



Bias Study Screener Bias Study Screener 

field respondents field nonrespondents 





Sample 


Standard 


Sample 


Standard 


Charaeteristie 


size Pereent 


Error 


size Pereent 


Error 



Household eharaeteristies 



Soeioeeonomie elassifieation 



Affluent or upper middle elass 
Middle-middle elass 


211 

1,303 


7.8 

51.0 


1.42 

3.09 


214 

1,117 


9.7 

53.8 


1.67 

2.53 


Working elass or poor 


1,003 


39.9 


3.12 


670 


33.8 


2.68 


Missing 


33 


1.2 


0.34 


49 


2.6 


0.74 


Evidenee of ehildren 


Yes 


590 


22.6 


1.09 


255 


12.0 


1.05 


No 


1,913 


75.6 


1.10 


1,717 


83.9 


1.31 


Missing 


47 


1.8 


0.39 


78 


4.1 


0.77 


Bars or gratings on doors or windows 


Yes 


193 


7.8 


2.23 


186 


9.3 


2.64 


No 


2,323 


90.9 


2.41 


1,794 


86.9 


2.65 


Missing 


34 


1.3 


0.31 


70 


3.8 


0.84 


Signs for private seeurity serviees 


Yes 


272 


10.4 


1.04 


268 


12.7 


1.29 


No 


2,237 


88.1 


1.32 


1,705 


83.1 


1.63 


Missing 


41 


1.5 


0.40 


77 


4.1 


0.75 


Bloek eharaeteristies 
Urbanieity 


Urban 


788 


32.2 


5.43 


670 


34.4 


5.90 


Suburban 


1,384 


54.0 


5.01 


1,157 


54.8 


5.41 


Rural 


351 


12.7 


2.62 


193 


9.3 


1.80 


Missing 


27 


1.1 


0.29 


30 


1.5 


0.60 


Traffie flow 


Light 


1,757 


67.9 


1.59 


1,373 


65.3 


3.83 


Moderate 


569 


22.9 


1.65 


470 


23.4 


2.61 


Heavy 


204 


8.4 


1.11 


183 


10.1 


1.77 


Missing 


20 


0.7 


0.21 


24 


1.2 


0.57 



See notes at end of table. 
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Table 8-4. Distributions of characteristics collected in the Interviewer Observation Form (lOF), 
for field respondents and field nonrespondents separately: NHES:2007 Bias Study — 
Continued 



Bias Study Screener Bias Study Screener 

field respondents field nonrespondents 





Sample 


Standard 


Sample 


Standard 


Charaeteristie 


size Pereent 


Error 


size Pereent 


Error 



Bloek eharaeteristies — eontinued 



Abandoned ears 



None 


2,171 


84.7 


1.83 


1,793 


87.6 


2.35 


Very few (roughly 1-2 ears) 


301 


12.4 


1.63 


205 


9.9 


1.91 


Some (roughly 3-4 ears) 


53 


2.0 


0.40 


22 


1.0 


0.30 


Many (roughly 5 or more ears) 


7 


0.3 


0.19 


2 


0.1 


0.05 


Missing 


18 


0.6 


0.14 


28 


1.4 


0.63 


Trash or junk 


None 


1,704 


66.9 


3.40 


1,465 


71.0 


2.88 


Very little 


582 


22.6 


2.42 


399 


19.6 


2.01 


Some 


208 


8.3 


1.21 


138 


6.7 


1.12 


A lot 


43 


1.8 


0.74 


20 


1.1 


0.42 


Missing 


13 


0.4 


0.12 


28 


1.5 


0.69 


Land use 


Primarily residential 


2,198 


86.2 


1.62 


1,768 


85.6 


1.45 


Primarily eommereial 


31 


1.2 


0.38 


24 


1.1 


0.33 


Primarily industrial 


3 


0.1 


0.05 


1 


0.0 


0.04 


Primarily vaeant lots or undeveloped spaee 


49 


1.7 


0.21 


31 


1.2 


0.28 


Mixed residential and eommereial 


125 


5.2 


1.04 


103 


5.7 


1.32 


Mixed residential and industrial 


10 


0.4 


0.23 


15 


0.7 


0.34 


Mixed residential and vaeant lots 


102 


3.9 


0.90 


57 


2.9 


0.92 


Other 


21 


0.8 


0.32 


25 


1.3 


0.29 


Missing 


11 


0.4 


0.12 


26 


1.3 


0.60 


Houses/apartments with window/door bars or 
gratings 


None 


1,956 


76.3 


4.48 


1,449 


70.7 


6.09 


Very few 


335 


13.2 


1.92 


299 


13.7 


2.85 


Some 


167 


6.9 


2.09 


177 


9.4 


2.46 


Most 


65 


2.6 


1.20 


86 


4.1 


1.56 


All 


8 


0.3 


0.20 


9 


0.5 


0.32 


Missing 


19 


0.7 


0.10 


30 


1.6 


0.60 



See notes at end of table. 
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Table 8-4. Distributions of characteristics collected in the Interviewer Observation Form (lOF), 
for field respondents and field nonrespondents separately: NHES:2007 Bias Study — 
Continued 



Bias Study Screener Bias Study Screener 

field respondents field nonrespondents 



Characteristic 


Sample 

size 


Percent 


Standard 

Error 


Sample 

size 


Percent 


Standard 

Error 


Block characteristics — continued 

Houses/apartments with signs for private 
security services 














None 


1,642 


64.7 


2.30 


1,109 


55.2 


3.14 


Very few 


568 


22.0 


1.71 


543 


25.0 


1.95 


Some 


245 


9.5 


0.97 


284 


13.9 


1.47 


Most 


59 


2.4 


0.68 


65 


3.2 


0.75 


All 


17 


0.7 


0.68 


18 


1.1 


0.69 


Missing 


19 


0.7 


0.15 


31 


1.6 


0.63 


Signs for neighborhood watch 


Yes 


309 


12.2 


2.89 


272 


12.8 


2.72 


No 


2,210 


86.7 


2.86 


1,736 


85.1 


2.99 


Missing 


31 


1.2 


0.24 


42 


2.1 


0.65 


Children 


Yes 


893 


34.3 


2.54 


690 


33.5 


4.08 


No 


1,628 


64.6 


2.56 


1,322 


64.6 


4.40 


Missing 

Interviewer heard non-English language 
spoken in neighborhood 


29 


1.1 


0.25 


38 


1.9 


0.57 


Yes 


246 


10.0 


2.16 


290 


15.0 


3.67 


No 

No people around or did not hear any 


1,149 


44.0 


4.84 


827 


39.3 


4.70 


language 


1,134 


45.3 


5.09 


904 


44.2 


5.28 


Missing 


21 


0.7 


0.21 


29 


1.5 


0.58 



NOTE: Shading indieates a signifieant differenee of 5 percentage points or more. Estimates were produced using base 
weights. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 



In addition to the interviewer-reported eharacteristies in the lOF, ZIP eode-level 
eharaeteristies were obtained from the Census 2000 Summary Files (SFs), and distributions of these ZIP 
code characteristics are given for field respondents and field nonrespondents (separately) in table 8-5. The 
characteristics given in table 8-5 were selected for consideration because they are available at the ZIP 
code-level from the Census 2000 SFs and are also available on the sampling frame used to select the RDD 
sample. There are some substantive differences between ZIP code-level characteristics of field 
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respondents and field nonrespondents. A higher percentage of field respondents than nonrespondents live 
in ZIP codes with lower median home values (74 percent versus 63 percent in the first through seventh 
deciles) and lower median income deciles (35 percent versus 28 percent in the first through third deciles). 



Table 8-5. Distributions of ZIP code-level characteristics, for field respondents and field 
nonrespondents separately: NHES:2007 Bias Study 



Characteristic 


Bias Study Screener field 
respondents 


Bias Study Screener field 
nonrespondents 


Sample 

size 


Percent 


Standard 

Error 


Sample 

size 


Percent 


Standard 

Error 


Census region 














Northeast 


557 


20.7 


3.09 


485 


22.9 


3.90 


Midwest 


486 


19.4 


3.00 


333 


15.4 


2.77 


South 


906 


35.0 


1.77 


696 


34.0 


2.63 


West 


601 


24.9 


1.77 


536 


27.8 


2.10 


Metro status 














In an MSA 


2,405 


94.7 


0.33 


1,950 


95.5 


0.48 


Not in an MSA 


145 


5.3 


0.33 


100 


4.5 


0.48 


Median home value 














1 st through 7th deciles 


1,902 


74.4 


4.67 


1,292 


62.7 


5.96 


8th through 10th deciles 


648 


25.6 


4.67 


758 


37.3 


5.96 


Median income 


1 st through 3rd deciles 


891 


35.3 


4.96 


546 


27.7 


4.29 


4th through 10th deciles 


1,659 


64.7 


4.96 


1,504 


72.3 


4.29 


Percent college graduates 














Less than 20 percent 


854 


33.6 


5.15 


556 


27.8 


5.23 


20 to 29 percent 


658 


25.8 


3.87 


574 


27.6 


3.63 


30 to 39 percent 


497 


19.5 


2.96 


403 


19.4 


2.76 


40 to 59 percent 


484 


18.7 


2.78 


449 


21.6 


3.16 


60 percent or more 


57 


2.4 


0.89 


68 


3.7 


1.46 


Percent White 














Less than 30 percent 


182 


7.5 


2.77 


255 


12.9 


5.19 


30 to 49 percent 


302 


12.4 


3.19 


206 


10.9 


3.10 


50 to 69 percent 


363 


14.8 


4.18 


336 


17.3 


4.06 


70 to 89 percent 


656 


26.3 


3.40 


474 


23.1 


3.81 


90 percent or more 


1,047 


39.1 


6.74 


779 


35.8 


5.48 


See notes at end of table. 
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Table 8-5. Distributions of ZIP code-level characteristics, for field respondents and field 
nonrespondents separately: NHES:2007 Bias Study — Continued 



Bias Study Screener field Bias Study Screener field 

respondents nonrespondents 



Characteristic 


Sample Percent 


Standard 


Sample Percent 


Standard 




size 


Error 


size 


Error 



Percent Black 



Less than 50 percent 


2,394 


93.6 


1.66 


1,931 


94.0 


1.27 


50 to 59 percent 


106 


4.5 


1.92 


62 


3.2 


1.35 


60 percent or more 


50 


1.9 


1.00 


57 


2.9 


1.57 


Percent Hispanic 


Less than 20 percent 


1,865 


72.2 


6.75 


1,364 


65.0 


7.37 


20 to 39 percent 


394 


15.8 


4.35 


365 


18.7 


4.82 


40 to 59 percent 


164 


6.6 


2.39 


132 


6.5 


2.26 


60 percent or more 


127 


5.4 


2.22 


189 


9.8 


4.11 


Percent renters 


Less than 30 percent 


864 


32.3 


4.75 


692 


32.2 


4.76 


30 to 59 percent 


1,389 


55.2 


4.84 


1,060 


51.9 


4.57 


60 percent or more 


297 


12.5 


4.16 


298 


15.9 


4.36 


Percent owners 


Less than 40 percent 


140 


5.5 


2.20 


171 


9.0 


3.00 


40 to 69 percent 


834 


34.6 


3.40 


667 


33.8 


3.70 


70 percent or more 


1,576 


59.9 


4.23 


1,212 


57.3 


5.07 



NOTE: Shading indicates a significant difference of 5 percentage points or more. Estimates were produced using base 
weights. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Surveys Program (NHES), 2007. 



Table 8-6 gives the distributions of eharaeteristies that were eolleeted in the Maximum Call 
posteard. For the Bias Study respondents (whether they responded to the telephone effort or responded as 
a result of the field follow-up), these items were eolleeted in the Sereener. For the field nonrespondents, 
these items were obtained from the posteard. Although two additional items — household size and highest 
education — were also asked on the postcard, the distributions of those two items are not presented here 
because, for respondents, these items are available only if at least one extended interview was completed 
in the household. 
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Table 8-6. Distributions of characteristics collected in the Maximum Call postcard: NHES:2007 Bias Study 



Bias Study field 

Bias Study TRC Bias Study field nonrespondents with MC 

Bias study respondents respondents respondents postcard 



Household characteristic 


Sample 

size 


Percent 


Standard 

error 


Sample 

size 


Percent 


Standard 

error 


Sample 

size 


Percent 


Standard 

error 


Sample 

size 


Percent 


Standard 

error 


Number of children under 18 


None 


3,120 


63.4 


1.10 


1,644 


70.1 


1.21 


1,476 


59.0 


1.42 


127 


71.0 


2.66 


1 


751 


15.8 


0.61 


287 


12.2 


0.68 


464 


18.1 


0.81 


26 


14.5 


2.57 


2-3 


916 


18.5 


0.90 


382 


16.3 


1.08 


534 


20.0 


1.05 


28 


13.7 


2.13 


4+ 


107 


2.3 


0.25 


31 


1.3 


0.32 


76 


2.9 


0.36 


2 






Home ownership 


Own 


3,570 


69.0 


1.53 


2,024 


86.3 


0.96 


1,546 


57.8 


1.98 


129 


64.9 


3.99 


Rent/other 


1,324 


31.0 


1.53 


320 


13.7 


0.96 


1,004 


42.2 


1.98 


54 


35.1 


3.99 



J Reporting standards not met. 

NOTE: Estimates were produced using base weights. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2007. 



As shown in table 8-6, a larger pereentage of telephone respondents than field respondents 
have no children under 18 (70 percent versus 59 percent). These results appear to be anomalous, in that all 
other evidence (in both the Bias Study and the NHES RDD surveys) has indicated that a higher 
percentage of households with children respond to the telephone Screener than households without 
children. However, the field respondents include cases with no matching telephone number or a 
mismatched number, not just TRC nonrespondents. Considering all Bias Study respondents (in both the 
TRC and the field), 63 percent have no children under 18, compared to 71 percent among field 
nonrespondents who returned the Maximum Call postcard. In terms of home tenure, a larger percentage 
of Bias Study telephone respondents than field respondents own their homes (86 percent versus 58 
percent), and 65 percent of field nonrespondents own their homes. 



8.2 An Examination of Noncoverage Bias 

Because the Bias Study sample is based on a sample of addresses, it includes persons 
residing in both telephone and nontelephone households. For this section, nontelephone households are 
defined as those without a landline so that the noncoverage bias associated with standard RDD surveys 
that do not include cell phones can be estimated. The NHES:2007 Bias Study survey instruments included 
a series of questions to capture the presence and number of telephone numbers in the household, so 
nontelephone households can be identified through the responses to these questions. 

In order to examine noncoverage bias in NHES:2007 estimates, fully weighted estimates 
from the full Bias Study sample (completed extended interviews from FT/RT and FTO cells of figure 4-1) 
were compared to estimates from the Bias Study sample restricted to telephone households (telephone 
household field respondents cell of figure 4-1). The weights for the telephone households were separately 
raked to population totals, since estimates of noncoverage bias in the Bias Study telephone household 
estimates are intended to provide an indication of noncoverage bias in the final RDD estimates, which 
included the raking adjustment. The estimates are tabulated for a set of characteristics in tables 8-7, 8-8, 
and 8-9, for the SR, PEI, and AEWR surveys, respectively. 
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Table 8-7. Estimates of noncoverage bias for various characteristics from the School Readiness (SR) Survey: NHES:2007 Bias Study 





Full treatment SR respondents 


With landline phone 




Bias 


1 


Relative bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate Standard 
(percent) error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Participation in center-based care 


177 


51.1 


2.29 


159 


53.4 


2.11 


2.4 


0.96 


4.5 


Specific skills 


Recognizes all colors 


249 


85.6 


2.34 


212 


84.4 


3.93 


-1.2 


2.21 


-1.4 


Counts to 20 or higher 


198 


68.4 


2.27 


170 


67.2 


3.22 


-1.2 


2.57 


-1.8 


Recognizes all letters 


93 


29.4 


2.89 


80 


28.9 


2.78 


-0.5 


1.87 


-1.7 


Writes first name 


193 


60.4 


3.08 


171 


62.9 


3.88 


2.5 


2.20 


4.1 


Holds a pencil 


252 


86.7 


1.90 


212 


85.6 


2.18 


-1.2 


1.65 


-1.3 


Speech is often understandable to a stranger 


249 


87.3 


2.17 


210 


87.0 


1.95 


-0.3 


1.39 


-0.3 


Reads or pretends to read storybooks 


289 


99.2 


0.45 


243 


99.0 


0.56 


-0.2 


0.14 


-0.2 


Parents believe it is essential to do certain things to 
prepare child for kindergarten 


Teach child the alphabet 


171 


57.7 


2.72 


149 


61.3 


2.73 


3.6 


1.54 


6.2 


Teach child about sharing 


177 


57.8 


2.97 


155 


61.4 


4.14 


3.6 


2.49 


6.2 


Teach child to read 


139 


48.0 


3.47 


119 


50.1 


4.64 


2.1 


2.70 


4.4 


Teach child numbers 


155 


51.7 


2.64 


136 


56.3 


3.00 


4.6 


2.52 


8.9 


Show child how to hold a pencil 


121 


37.9 


2.88 


106 


40.3 


3.56 


2.4 


2.56 


6.3 


Family member read to child everyday in the past week 


169 


56.9 


4.02 


142 


54.9 


5.56 


-2.1 


3.15 


-3.5 


Parents report usually doing certain reading-related 
activity with child 


Ask child what is in a picture 


159 


52.9 


4.00 


134 


52.8 


4.40 


-0.1 


3.00 


-0.2 


Stop reading and point out letters 


95 


31.9 


4.14 


85 


34.5 


4.22 


2.5 


3.42 


8.2 


Ask child to read with parent 


84 


29.7 


3.05 


71 


27.0 


3.50 


-2.7 


3.17 


-9.1 


Talk about the story and what happened 


165 


60.3 


3.18 


137 


58.0 


4.02 


-2.3 


2.41 


-3.8 


Parents did home activities with child in the past week^ 


123 


41.7 


3.23 


101 


38.9 


3.39 


-2.8 


2.01 


-6.7 


Parents took 3 or more outings with child in the past month"* 


138 


46.0 


3.71 


114 


44.4 


4.08 


-1.7 


2.55 


-3.5 



See notes at end of table. 



Table 8-7. Estimates of noncoverage bias for various characteristics from the School Readiness (SR) Survey: NHES:2007 Bias Study — 
Continued 



Relative 

Full treatment SR respondents With landline phone Bias* bias^ 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Child watches 2 or more hours of TV in a typical weekday 


179 


71.1 


2.10 


145 


67.4 


2.50 


-3.6 


1.87 


-5.2 


Child has a disability 


37 


14.1 


3.34 


32 


14.5 


4.54 


0.4 


2.20 


2.8 


Child’s age 


3 years 


128 


43.9 


2.86 


102 


41.5 


3.61 


-2.4 


2.20 


-5.5 


4 years 


116 


37.3 


2.64 


101 


38.2 


3.81 


0.9 


2.05 


2.4 


5 years and older 


48 


18.8 


3.08 


43 


20.3 


4.48 


1.6 


2.19 


8.0 


Child’s sex 


Male 


169 


62.3 


2.97 


137 


60.8 


3.78 


-1.6 


2.16 


-2.4 


Female 


123 


37.7 


2.97 


109 


39.2 


3.78 


1.6 


2.16 


4.0 


Flousehold urbanicity 


Urban 


247 


80.8 


1.62 


206 


80.6 


1.65 


-0.2 


0.22 


-0.2 


Rural 


45 


19.2 


1.62 


40 


19.4 


1.65 


0.2 


0.22 


1.0 


Flome tenure 


Own 


201 


62.1 


0.87 


175 


62.6 


0.62 


0.5 


1.02 


0.8 


Rent/other 


91 


37.9 


0.87 


71 


37.4 


0.62 


-0.5 


1.02 


-1.3 


Parents’ educational attainment 


Fligh school diploma or below 


91 


34.5 


3.80 


63 


25.3 


4.13 


-9.2 


2.47 


-26.7 


Beyond high school diploma 


201 


65.5 


3.80 


183 


74.7 


4.13 


9.2 


2.47 


14.0 


Parents’ language 


Both/only parent(s) speak(s) English 


254 


88.0 


2.48 


214 


88.9 


1.90 


0.9 


1.74 


1.0 


One of two parents speaks English 


t 






t 


t 


t 


t 






No parent speaks English 


29 


11.0 


2.60 


25 


10.7 


1.80 


-0.3 


1.82 


-2.7 



See notes at end of table. 




Table 8-7. Estimates of noncoverage bias for various characteristics from the School Readiness (SR) Survey: NHES:2007 Bias Study — 
Continued 





Full treatment SR respondents 


With landline phone 




Bias 


1 


Relative bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate Standard 

(percent) error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Mothers’ employment status 


35 hours or more per week 


ill 


41.6 


3.34 


94 


40.5 


3.69 


-1.1 


2.16 


-2.6 


Less than 35 hours per week 


69 


20.4 


3.21 


62 


23.2 


3.79 


2.8 


1.18 


13.7 


Looking for work 


23 


10.0 


2.49 


15 


8.2 


2.33 


-1.7 


1.42 


-18.0 


Not in labor force 


83 


26.1 


2.65 


70 


25.9 


2.78 


-0.2 


2.35 


-0.8 


No mother in household 


6! 


1.9! 


1.18! 


5! 


2.2! 


1.04! 


0.3! 


0.33! 


15.8! 


Poverty status 


Poor 


80 


24.5 


2.19 


63 


24.6 


3.00 


0.1 


1.60 


0.4 


Nonpoor 


212 


75.5 


2.19 


183 


75.4 


3.00 


-0.1 


1.60 


-0.1 


Household income 


Less than $15,000 


52 


15.3 


1.71 


41 


13.9 


2.69 


-1.4 


1.91 


-9.2 


$15,001 to $30,000 


62 


17.6 


2.39 


51 


20.5 


3.60 


2.9 


2.54 


16.5 


$30,001 to $50,000 


50 


17.6 


2.77 


41 


15.6 


2.73 


-2.0 


1.34 


-11.4 


More than $50,000 


128 


49.5 


2.92 


113 


50.0 


3.48 


0.5 


1.15 


1.0 


Family structure 


Mother and father 


219 


73.4 


4.36 


187 


72.8 


4.20 


-0.7 


1.96 


-0.8 


Mother 


60 


22.4 


3.09 


48 


22.3 


3.60 


-0.1 


2.17 


-0.4 


Father 


6! 


1.9! 


1.18! 


5! 


2.2 


1.04 


0.3 


0.33 


15.8! 


Nonparent guardian(s) 


7 


2.3 


0.96 


6 


2.8 


1.17 


0.5 


0.77 


21.7 



See notes at end of table. 

! Interpret with caution, 
t Reporting standards not met. 

’ Bias is estimated as the difference between the “with landline phone” SR respondent estimate and the full treatment SR respondent estimate. 

^ Relative bias is estimated as the bias estimate divided by the full treatment SR respondent estimate, and is expressed as a percentage. 

^ Told child a story; taught child letters, words, or numbers; taught child songs or music; did arts and crafts with child; played sports, active games or exercised together; and played 
board games or did puzzles with child. 

^ Any three or more of the following: Visited a library; visited a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or historical site; visited a zoo 
or aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting event (outside of school) in which the child was not a 
player. 

NOTE: Shading indicates a significant difference of 5 percentage points or more. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, School Readiness (SR) Survey of the National Household Education Surveys Program, 2007. 







Table 8-8. Estimates of noncoverage bias for various characteristics from the Parent and Family Involvement (PFI) Survey: NHES:2007 
Bias Study 





Full treatment PFI respondents 


With landline phone 




Bias 


1 


Relative bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate Standard 
(percent) error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Parents participate in 5 or more activities in the 


student’s schoor 


566 


52.1 


2.22 


500 


54.6 


2.67 


2.6 


1.18 


4.8 


Parents report school provides information very well 


About how student is doing in school 


668 


58.2 


2.20 


582 


60.8 


1.81 


2.6 


1.34 


4.5 


About how to help student with his/her homework 


525 


47.9 


2.07 


455 


49.0 


2.39 


1.2 


1.22 


2.3 


About why student is placed in particular groups of 


classes 


496 


45.6 


1.67 


432 


46.7 


2.27 


1.0 


1.26 


2.4 


About how to help student plan for college or 


vocational school 


237 


38.9 


1.65 


210 


39.1 


1.94 


0.1 


1.06 


0.5 


About the family’s expected role at student’s school 


539 


46.9 


1.93 


473 


48.2 


2.34 


1.3 


1.37 


2.8 


Parent reports being very satisfied with 4 or more 


aspects of the student’s school"* 


798 


73.7 


1.87 


685 


73.2 


2.17 


-0.4 


1.00 


-0.7 


Parents participated in 5 or more home learning activities^ 


229 


42.2 


2.28 


197 


45.3 


2.62 


3.1 


1.62 


7.3 


Parents took 3 or more outings with student in the past 


month® 


559 


50.3 


2.09 


494 


52.2 


2.25 


1.9 


1.14 


3.8 


Parents check to see that student’s homework gets done 


885 


87.9 


0.99 


754 


87.6 


1.04 


-0.2 


0.54 


-0.3 


Parents received information about free tutoring 


465 


42.6 


2.15 


395 


43.0 


2.55 


0.4 


1.27 


0.9 


Parent expects student to earn a college degree or higher 


424 


67.2 


2.87 


380 


69.7 


2.98 


2.5 


1.79 


3.7 


Family plans to help pay for student’s education after 


high school 


475 


83.1 


2.19 


420 


82.1 


2.62 


-1.0 


1.24 


-1.2 


Student participated in school activities 


624 


55.8 


1.89 


550 


56.9 


2.27 


1.1 


1.25 


2.0 


Student has a disability 


257 


23.0 


2.16 


213 


21.9 


1.70 


-1.1 


1.09 


-4.8 



See notes at end of table. 





Table 8-8. Estimates of noncoverage bias for various characteristics from the Parent and Family Involvement (PFI) Survey: NHES:2007 
Bias Study — Continued 





Full treatment PFI respondents 


With landline phone 




Bias 


1 


Relative hias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate Standard 
(percent) error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Student’s sex 


Male 


575 


51.8 


2.10 


484 


49.9 


2.46 


-1.8 


1.23 


-3.7 


Female 


548 


48.2 


2.10 


481 


50.1 


2.46 


1.8 


1.23 


3.9 


Flousehold urbanicity 


Urban 


933 


81.8 


0.06 


796 


81.7 


0.11 


-0.1 


0.06 


-0.1 


Rural 


190 


18.2 


0.06 


169 


18.3 


0.11 


0.1 


0.06 


0.5 


Flome tenure 


Own 


837 


69.7 


0.80 


743 


70.2 


0.67 


0.5 


0.45 


0.7 


Rent/other 


286 


30.3 


0.80 


222 


29.8 


0.67 


-0.5 


0.45 


-1.7 


Parents’ educational attainment 


Fligh school diploma or below 


326 


31.7 


2.31 


259 


29.4 


2.38 


-2.4 


1.05 


-7.3 


Beyond high school diploma 


797 


68.3 


2.31 


706 


70.6 


2.38 


2.4 


1.05 


3.4 


Parents’ language 


Both/only parent(s) speak(s) English 


1012 


90.9 


1.29 


884 


91.8 


1.34 


0.9 


0.74 


1.0 


One of two parents speaks English 


22 


2.1 


0.61 


17 


2.1 


0.78 


# 


0.39 


# 


No parent speaks English 


66 


7.0 


1.17 


49 


6.0 


1.23 


-1.0 


0.66 


-14.3 


Mothers’ employment status 


35 hours or more per week 


521 


47.0 


2.12 


448 


47.1 


2.33 


0.1 


1.26 


0.2 


Less than 35 hours per week 


260 


21.1 


1.68 


235 


22.8 


1.91 


1.8 


0.91 


8.1 


Looking for work 


45 


4.1 


0.56 


35 


3.6 


0.69 


-0.4 


0.54 


-12.2 


Not in labor force 


241 


23.4 


1.24 


200 


21.9 


2.17 


-1.5 


1.79 


-6.4 


No mother in household 


56 


4.4 


0.55 


47 


4.5 


0.64 


0.1 


0.45 


2.3 


Poverty status 


Poor 


221 


19.7 


1.05 


173 


20.5 


1.11 


0.7 


0.59 


4.1 


Nonpoor 


902 


80.3 


1.05 


792 


79.5 


1.11 


-0.7 


0.59 


-1.0 



See notes at end of table. 






Table 8-8. Estimates of noncoverage bias for various characteristics from the Parent and Family Involvement (PFI) Survey: NHES:2007 
Bias Study — Continued 





Full treatment PFI respondents 


With landline phone 




Bias 


1 


Relative bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate Standard 
(percent) error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Household income 


Less than $15,000 


153 


12.9 


0.64 


115 


12.0 


0.73 


-0.9 


0.67 


-7.0 


$15,001 to $30,000 


179 


15.2 


1.22 


144 


16.2 


1.30 


0.9 


1.03 


6.6 


$30,001 to $50,000 


171 


16.6 


2.23 


142 


15.9 


2.49 


-0.7 


0.54 


-4.2 


More than $50,000 


620 


55.3 


2.22 


564 


55.9 


2.50 


0.6 


0.78 


1.1 


Family structure 


Mother and father 


823 


74.1 


1.88 


723 


74.6 


2.30 


0.6 


1.08 


0.7 


Mother 


204 


18.6 


1.61 


164 


17.9 


2.15 


-0.7 


1.01 


-3.8 


Father 


54 


4.0 


0.42 


47 


4.5 


0.64 


0.4 


0.41 


12.5 


Nonparent guardian(s) 


42 


3.3 


0.55 


31 


3.0 


0.67 


-0.3 


0.71 


-9.1 


School type 


Public 


989 


89.9 


1.65 


845 


89.0 


1.62 


-0.9 


0.62 


-1.0 


Private 


114 


10.1 


1.65 


103 


11.0 


1.62 


0.9 


0.62 


8.9 


School size 


Under 300 


167 


14.6 


1.51 


144 


14.4 


1.75 


-0.2 


0.91 


-1.4 


300-599 


321 


28.8 


2.59 


275 


29.6 


2.49 


0.8 


1.11 


2.8 


600-999 


286 


25.8 


1.81 


250 


26.6 


1.56 


0.9 


0.71 


3.1 


1 ,000 or more 


323 


30.8 


1.65 


274 


29.4 


1.86 


-1.4 


0.76 


-4.5 



# Rounds to zero. 

' Bias is estimated as the difference between the “with landline phone” PFI respondent estimate and the full treatment PFI respondent estimate. 

^ Relative bias is estimated as the bias estimate divided by the full treatment PFI respondent estimate, and is expressed as a percentage. 

^ Any five or more of the following: Attended a general school meeting; attended a meeting of the parent-teacher organization or association; went to a regularly scheduled parent- 
teacher conference with the student’s teacher; attended a school or class event because of the student; served as a volunteer in the student’s classroom or elsewhere in the school; 
participated in fundraising for the school; served on a school committee; and met with a guidance counselor in person. 

** Any four or more of the following: School student attends this year; teachers student has this year; academic standards of the school; order and discipline at the school; and way 
that school staff interact with parents. 

^ Any five or more of the following: Told student a story; did arts and crafts with student; played sports, active games or exercised together; worked on projects such as building, 
making, or fixing something with student not as a chore; talked with student about family history or ethnic heritage; and played board games or did puzzles with student. 

® Any three or more of the following: Visited a library; a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or historical site; visited a zoo or 
aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting event (outside of school) in which the student was not a player. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement in Education (PFI) Survey of the National Household Education 
Surveys Program, 2007. 



Table 8-9. Estimates of noncoverage bias for various characteristics from the Adult Education for Work-Related Reasons (AEWR) 
Survey: NHES:2007 Bias Study 





Full treatment AEWR 
respondents 


With landline phone 




Bias 


1 


Relative 

bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate Standard 
(percent) error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Participates in adult education for work-related reasons 


495 


39.1 


1.76 


395 


37.1 


2.14 


-1.9 


1.57 


-5.4 


Participates in employer-supported AEWR 


347 


70.0 


3.57 


280 


69.5 


3.18 


-0.5 


2.69 


-0.7 


Participates in distance education 


285 


60.1 


2.98 


234 


62.1 


2.85 


2.1 


2.28 


3.2 


Participates in program to earn a college or university 


degree 


124 


9.4 


0.81 


82 


8.5 


1.04 


-1.0 


0.80 


-10.6 


Participates in program to earn a vocational or 


technical diploma 


56 


4.2 


0.70 


46 


4.2 


1.06 


# 


0.61 


# 


Participates in formal apprenticeship program 


5! 


0.5! 


0.33! 


3! 


0.3! 


0.14! 


-0.2! 


0.31! 


-40.0! 


Participates in work-related training or courses 


416 


33.2 


1.49 


336 


31.0 


1.89 


-2.2 


1.41 


-7.1 


Participates in 4 or more informal work-related 


learning activities^ 


324 


29.2 


2.25 


275 


31.4 


2.69 


2.2 


1.20 


7.0 


Has any condition that limits ability to work 


153 


15.2 


2.63 


126 


14.8 


2.80 


-0.4 


1.23 


-2.7 


Age 


1 6 to 24 years 


104 


13.1 


1.17 


60 


12.7 


1.61 


-0.4 


0.85 


-3.1 


25 to 34 years 


168 


17.5 


1.86 


119 


17.8 


2.55 


0.3 


1.35 


1.7 


35 to 44 years 


131 


16.9 


1.31 


112 


18.4 


1.40 


1.5 


0.91 


8.2 


45 to 54 years 


217 


21.4 


1.75 


184 


18.9 


1.29 


-2.5 


1.18 


-13.2 


55 years and older 


445 


31.1 


1.20 


411 


32.2 


1.15 


1.1 


0.56 


3.4 



See notes at end of table. 



Table 8-9. Estimates of noncoverage bias for various characteristics from the Adult Education for Work-Related Reasons (AEWR) 
Survey: NHES:2007 Bias Study — Continued 





Full treatment AEWR 
respondents 


With landline phone 




Bias 


1 


Relative 

hias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate Standard 
(percent) error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Census region 


Northeast 


259 


25.7 


4.68 


231 


25.1 


5.33 


-0.6 


1.35 


-2.4 


Midwest 


235 


21.3 


2.89 


203 


21.5 


3.06 


0.2 


0.86 


0.9 


South 


314 


32.8 


3.28 


242 


33.2 


3.83 


0.4 


0.97 


1.2 


West 


257 


20.2 


0.00 


210 


20.2 


# 


# 


# 


# 


Home tenure 


Own 


794 


70.1 


0.79 


709 


71.1 


0.66 


1.0 


0.61 


1.4 


Rent/other 


271 


29.9 


0.79 


177 


28.9 


0.66 


-1.0 


0.61 


-3.5 


Household size 


1 person 


278 


9.6 


0.65 


211 


9.2 


0.88 


-0.4 


0.64 


-4.3 


More than 1 person 


787 


90.4 


0.65 


675 


90.8 


0.88 


0.4 


0.64 


0.4 


Marital status 


Never married 


249 


24.2 


1.42 


162 


22.8 


2.05 


-1.3 


1.28 


-6.1 


Currently married 


531 


55.9 


2.06 


481 


58.3 


2.26 


2.4 


1.41 


4.1 


Other 


285 


20.0 


1.69 


243 


18.9 


1.92 


-1.1 


1.33 


-5.8 


Household income 


Less than $15,000 


194 


10.6 


0.54 


137 


10.5 


0.93 


-0.1 


0.68 


-1.0 


$15,001 to $30,000 


212 


16.1 


1.24 


173 


16.5 


1.78 


0.4 


1.25 


2.4 


$30,001 to $50,000 


188 


23.1 


1.59 


151 


20.7 


1.79 


-2.3 


1.44 


-11.6 


More than $50,000 


471 


50.3 


1.30 


425 


52.3 


1.26 


2.1 


1.51 


3.8 



See notes at end of table. 



Table 8-9. Estimates of noncoverage bias for various characteristics from the Adult Education for Work-Related Reasons (AEWR) 
Survey: NHES:2007 Bias Study — Continued 





Full treatment AEWR 
respondents 


With landline phone 




Bias 


1 


Relative 

bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate Standard 
(percent) error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Language spoken most at home 


English 


999 


92.4 


1.60 


837 


93.1 


1.49 


0.7 


0.60 


0.8 


Spanish 


39 


4.7 


1.08 


31 


4.9 


1.22 


0.2 


0.47 


4.1 


Other language 


18 


2.1 


0.89 


14 


1.6 


0.68 


-0.5 


0.45 


-31.3 


English and Spanish equally 


8 


0.7 


0.31 


3! 


0.2! 


0.17! 


-0.4! 


0.26! 


-57.1! 


English and other language equally 


















t 


Employment status 


Employed 


627 


61.2 


1.93 


510 


61.1 


1.45 


-0.1 


1.54 


-0.2 


Unemployed but looking for work 


44 


5.0 


1.28 


34 


6.1 


1.57 


1.1 


0.66 


18.0 


Not in the labor force 


394 


33.8 


1.35 


342 


32.8 


1.52 


-1.0 


1.60 


-3.0 



# Rounds to zero. 

! Interpret with caution. 

J Reporting standards not met. 

’ Bias is estimated as the difference between the “with landline phone” AEWR respondent estimate and the full treatment AEWR respondent estimate. 

^ Relative bias is estimated as the bias estimate divided by the full treatment AEWR respondent estimate, and is expressed as a percentage. 

^ Any four or more of the following: Received on-the-job demonstrations of equipment, techniques, or procedures by a supervisor or coworker; received other supervised training 
or mentoring on the job; self-paced study using books, procedures manuals, audio tapes, or videos; self-paced study using computer-based software tutorials; attended “brown-bag” 
or informal presentations; attended conferences, trade shows, or conventions related to the adult’s work or profession; and read professional journals, trade publications, or work- 
related magazines. 

NOTE: Shading indicates a significant difference of 5 percentage points or more. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Adult Education for Work-Related Reasons Survey of the National Household Education 
Surveys Program, 2007. 



For the SR Survey, table 8-7 shows that the percentage of preschoolers in telephone 
households whose parents’ highest educational attainment is beyond a high school diploma (75 percent) is 
higher than the overall percentage of preschoolers whose parents’ highest educational attainment is 
beyond a high school diploma (66 percent). The key estimates for the SR survey, however, are 
comparable for preschoolers in telephone households and the overall sample. Although there are some 
statistically significant differences between school-age children in telephone households and school-age 
children overall (table 8-8), as well as between adults in telephone households and adults overall (table 8- 
9), none of these differences meet the criterion for substantive importance of at least 5 percentage points. 
As shown in section 7.3, some differences were found between RDD estimates before and after the raking 
adjustment, indicating potential noncoverage bias in the pre-raked estimates. Based on the analysis in 
section 7.3 and the analysis in this section, the raking adjustments appear to be effective in reducing 
noncoverage bias in the final estimates. 



171 




This page is intentionally blank. 




9. DISCUSSION 



With survey response rates — in particular, random digit dial (RDD) survey response rates — 
on the decline, the potential for nonresponse bias is an important concern to survey methodologists and 
data analysts. Response rates are frequently used as a measure of the potential for nonresponse bias. A 
general perception is that surveys with higher response rates have less potential for nonresponse bias than 
surveys with lower response rates. Another important attribute is the face validity of the survey that is 
diminished if a high survey response rate is not obtained. In addition, there is increasing concern over 
noncoverage bias in RDD surveys, as the rate of cell-phone-only households continues to rise. 

Several analyses have been conducted to evaluate nonresponse and noncoverage bias in the 
NHES surveys. As noted in chapter 1, estimates from the NHES:2007 RDD sample and earlier NHES 
surveys have been compared to other surveys with higher response rates. These comparisons are valuable, 
but any differences might be due to a variety of causes other than nonresponse and noncoverage. For 
example, different questionnaire wordings, modes, and context effects could be responsible for 
differences. The source of the difference is important because remedial actions (such as using frames that 
cover a higher proportion of the population) might not have any effect on the differences due to these 
other sources. 

Another nonresponse bias approach used a multivariate Chi-Square Automatic Interaction 
Detection (CHAID) analysis to detect differences in response rates among subgroups (described in 
chapter 5). These results were evaluated for consistency between years. A limitation of this approach is 
that it only provides indicators of potential nonresponse bias rather than direct estimates. 

In addition, an extensive evaluation of nonresponse bias was performed for the NHES:2001 
(Montaquila et al. 2008). The goal of that research was to investigate nonresponse bias in the estimates if 
the survey had used different data collection procedures that would have resulted in lower response rates. 
A limitation of this analysis is that the evaluation was limited to the original set of respondents. Through 
such studies, researchers are able to examine bias due to excluding hard-to-contact or reluctant 
respondents, thus informing decisions about how much additional effort might be needed to reduce 
nonresponse bias. On the other hand, direct assessment of nonresponse bias due to cases that did not 
complete the survey was not possible. 
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While each of these methods of analyzing bias has its own limitations, it has been 
encouraging that none of the efforts has indicated nonresponse bias of substantive importance in the 
estimates. Many of these approaches essentially used different resources and methodologies, so the 
collective evaluation provides a more robust estimate of the lack of substantively important bias than any 
one of them alone could. Nevertheless, the falling response and coverage rates in RDD surveys prompted 
further research into the potential for bias. 

The NHES:2007 Bias Study was developed to provide a more direct assessment of 
nonresponse bias for cases that did not complete the survey by telephone. This study was the first full- 
scale NHES effort undertaken to follow up with the nonrespondents in the field. The in-field follow-up 
was successful in yielding additional completes. Of cases that refused the Screener on the telephone, 42 
percent completed the Screener in the field. Similarly, 46 percent of telephone maximum call cases and 
45 percent of telephone noncontact cases completed the Screener during the in-field follow-up. This study 
is still limited because survey estimates for the households that did not respond to either the telephone or 
in-field effort are not available. However, general characteristics of the cases that failed to respond to the 
field effort were available from the lOF and decennial census. A higher proportion of field respondents 
than field nonrespondents were found to live in ZIP codes with lower median home values and lower 
median income deciles. Also, interviewers classified a higher proportion of field respondents as living in 
working class or poor households, having evidence of children, and being on blocks where no households 
had signs for private security, compared to field nonrespondents. For the majority of characteristics 
examined, however, the field respondents were found to be similar to field nonrespondents. 

Results from this study suggest that there is no systematic pattern of bias in key statistics 
from the NHES:2007. A comparison of the fully weighted RDD estimates to Bias Study estimates 
showed potential for bias in five estimates. The estimated percentages of preschoolers who count to 20 or 
higher, whose speech is often understandable to a stranger, and who watch two or more hours of TV in a 
typical weekday were lower for the RDD survey than the Bias Study. The estimates of the percentage of 
preschoolers whose mother is not in the labor force, as well the percentage of adults who are currently 
married, were higher for the RDD survey than the Bias Study. However, the majority of estimates 
evaluated showed no evidence of bias of substantive importance. 

In addition to the evaluation of overall bias in the NHES:2007 estimates, the Bias Study also 
allowed for the estimation of the nonresponse and noncoverage bias components. The NHES:2007 

The percentage includes nonhostile refusals only. Hostile refusals were not sent to the field. 



174 




estimates were produced using weights that were adjusted for nonresponse and calibrated to population 
totals — adjustments that are expected to reduce nonresponse and noncoverage bias. The results from the 
NHES:2007 Bias Study, in concert with the previous bias analyses, suggest that despite the falling 
response rates, there is no bias of substantive importance in the NHES:2007 estimates due to 
nonresponse. A comparison of estimates before and after the raking adjustments indicated potential 
noncoverage biases in some unadjusted SR survey outcome estimates, as well as in some demographic 
estimates, that were reduced through the weighting process (see section 7.3 for a complete list). While 
the weighting adjustments appear to have reduced noncoverage bias, the Bias Study analysis did provide 
evidence of the potential for noncoverage bias in the final estimate of the percentage of preschoolers 
whose parents’ highest educational attainment is beyond a high school diploma. Although estimates of 
noncoverage bias in other final estimates examined in this study are not of substantive importance as 
defined for this report, noncoverage bias may become more of an issue in the future as more households 
drop their landline telephone service. 
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U.S. DEPARTMENT OF EDUCATION 

INSTITUTE OF EDUCATION SCIENCES 

NATIONAL CENTER FOR EDUCATION STATISTICS 



December, 2006 



Dear Sir or Madam: 

The National Center for Education Statistics, part of the United States Department of Education, needs 
your help with an important education research study. The National Household Education Surveys Program 
(NHES) will be conducted in households all over the country to learn about educational experiences of both adults 
and children — important issues we can only learn about by speaking with people like you. We have enclosed a 
token of our appreciation. 

Your participation in this research study will help us learn about 

• The early educational experiences of young children and how families prepare for children to start 
school; 

• The ways the parents and families are involved in children’s education at school and at home, and how 
schools work with families; and 

• Types of education and training that adults may take part in. 

Our study uses a scientific sample of households in the country. Some households were selected from 
among all possible telephone numbers and some were selected from address lists. Your household represents 
thousands of other households. Even if there are no children or adults who have taken part in educational activities 
in your household, it is important that we talk to you so that the study results accurately reflect the experiences of 
all children and adults across the nation. 



Please be assured that all information you give is completely confidential and will never be published with 
your name. More details about the interviews, how your household was selected, and how to obtain reports from 
previous surveys are provided on the back of this letter. 

Westat, a social science research firm, will conduct this study. An interviewer will call you sometime 
between January 2 and April 1, 2007. A few initial questions will determine if someone in your household is 
selected for an interview. If we happen to call at an inconvenient time, please suggest a time that is better for you. 
If you would like to set an appointment before we call, you can contact Westat at their toll-free number (1-888- 
696-5670) and give your telephone number and the time that is convenient for you. 



Please help us in our efforts to better understand education in the United States. We recognize that you 
have many demands on your time, and we thank you in advance for your cooperation in this important research. 



Sincerely, 






Gail Mulligan 
Project Officer 

National Household Education Surveys Program 



WASHINGTON, D.C. 20006 
A-1 




Some Frequently Asked Questions about the 
National Household Education Surveys Program (NHES) 



Q. How will the study results be used? What will you do with this information? 

A. The information we collect will be used to better understand educational experiences and needs. 

Findings will be published in U.S. Department of Education reports. Reports from NFIES surveys 
are available online at http://nces.ed.gov/nhes or by writing to the National Center for Education 
Statistics at the address below. The NFIES reports, which do not reveal individual answers but 
rather grouped data for large numbers of people, are widely distributed to educators, researchers, 
policy makers, news organizations, and the general public. 

Q. How did you get my phone number and address? 

A. Our study uses a random sample of households in the country. Some households were selected 
from among all possible telephone numbers and some were selected from address lists. We do 
not use telephone directories to select telephone numbers. If your number was unlisted, it still is. 

Once our sample was selected, an independent organization matched a list of published addresses 
and phone numbers. This letter was sent to every address that was matched with a telephone 
number. Address information is kept confidential and will be destroyed as soon as the data 
collection is completed. 

Q. Will you keep my information confidential? 

A. All information you give to the interviewer will be kept completely confidential. Employees of the 
U.S. Department of Education and Westat who are working on this study are required by law to 
protect the confidentiality of respondents. Your responses will be protected to the fullest extent 
allowable under law. Also, individual responses are never published in reports; they are combined 
with the responses of others and are published as grouped data only. 

Q. How long will the survey take? 

A. First, there are a few short questions to see if any members of your household qualify for the study. 
They take about 4 minutes. In about half of all households, no one is selected for an interview. If 
someone is chosen for an interview, it will take approximately 15 to 30 minutes, on average, 
depending on the interview. 

Q. What is the authority for conducting this survey? 

A. This study has been approved by the Office of Management and Budget, the office that reviews 
all federally sponsored surveys. The approval number assigned to this study is 1850-0768. You 
may send any comments about this survey, including its length, to the Federal Government. 
Write to Gail Mulligan, National Center for Education Statistics, U.S. Department of Education, 
1990 K Street NW, Room 9065, Washington, DC 20006-5650. You may send e-mail to 
nhes@ed.gov. 
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U.S. DEPARTMENT OF EDUCATION 

INSTITUTE OF EDUCATION SCIENCES 

NATIONAL CENTER FOR EDUCATION STATISTICS 



Dear Sir or Madam: 



February, 2007 



Recently, a professional telephone interviewer called your household for a national research study 
about education. The 2007 National Flousehold Education Surveys Program (NFIES) is sponsored by the 
National Center for Education Statistics, part of the United States Department of Education. As of the date 
we mailed this letter, we had not completed an interview with your household. You may have already 
received a letter from me introducing NFIES. I am writing to give you more information about the study. I 
hope that after reading it, you will take part in this important research effort. We have enclosed a token of our 
appreciation. 

The purpose of this research is to learn about the educational experiences of both adults and children. 
We can only learn about these issues by speaking to families like yours. Your participation in the 2007 
NFIES will help us learn about 



• The early educational experiences of young children and how families prepare for children to start 
school; 

• The ways the parents and families are involved in children’s education at school and at home, and 
how schools work with families; and 

• Types of education and training that adults may take part in. 

Our study uses a scientific sample of households in the country. Some households were selected from 
among all possible telephone numbers and some were selected from address lists. Your household represents 
thousands of other households. Even if no one in your household has taken part in educational activities, it is 
important that we talk to you so that the study results can reflect the experiences of all children and adults across 
the nation. Please be assured that the information you provide is completely confidential and will never be given 
out with your name. 

Westat, a social science research firm, is conducting this study. In the next week or two, an 
interviewer from Westat will call your household again. If we happen to call at an inconvenient time, please 
suggest a time that is better for you. If you would like to set an appointment before we call, you can contact 
Westat at their toll-free number (1-888-696-5670) and give your telephone number and the time that is 
convenient for you. 

We know that you have many demands on your time, and we thank you in advance for your help in this 
important research effort to better understand education in the United States. 



Sincerely, 






Gail Mulligan 
Project Officer 

National Household Education Surveys Program 



WASHINGTON, D.C. 20006 
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Some Frequently Asked Questions about the 
National Household Education Surveys Program (NHES) 

Q. How will the study results be used? What will you do with this information? 

A. The information we collect will be used to better understand educational experiences and needs. Findings 
will be published in U.S. Department of Education reports. Reports from NFIES surveys are available 
online at http://nces.ed.gov/nhes or by writing to the National Center for Education Statistics at the address 
below. The NFIES reports, which do not reveal individual answers but rather grouped data for large 
numbers of people, are widely distributed to educators, researchers, policy makers, news organizations, and 
the general public. 

Q. How did you get my phone number and address? 

A. Our study uses a random sample of households in the country. Some households were selected from 
among all possible telephone numbers and some were selected from address lists. We do not use 
telephone directories to select telephone numbers. If your number was unlisted, it still is. 

Once our sample was selected, an independent organization matched a list of published addresses and phone 
numbers. This letter was sent to every address that was matched with a telephone number. Address 
information is kept confidential and will be destroyed as soon as the data collection is completed. 

Q. Will you keep my information confidential? 

A. All information you give to the interviewer will be kept completely confidential. Employees of the U.S. 
Department of Education and Westat who are working on this study are required by law to protect the 
confidentiality of respondents. Y our responses will be protected to the fullest extent allowable under law. 
Also, individual responses are never published in reports; they are combined with the responses of others 
and are published as grouped data only. 

Q. How long will the survey take? 

A. First, there are a few short questions to see if any members of your household qualify for the study. They 
take about 4 minutes. In about half of all households, no one is selected for an interview. If someone is 
chosen for an interview, it will take approximately 20 to 35 minutes, on average, depending on the 
interview. 

Q. What is the authority for conducting this survey? 

A. This study has been approved by the Office of Management and Budget, the office that reviews all 
federally sponsored surveys. The approval number assigned to this study is 1850-0768. You may send 
any comments about this survey, including its length, to the Federal Government. Write to Gail 
Mulligan, National Center for Education Statistics, U.S. Department of Education, 1990 K Street NW, 
Room 9065, Washington, DC 20006-5650. You may send e-mail to nhes@ed.gov. 

Q. I got a letter earlier. Why did I get another letter? 

A. Sometimes letters do not get delivered or are not read for a number of reasons. This letter is sent to assure 
that someone in your household has a chance to read information about the study. 
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U.S. DEPARTMENT OF EDUCATION 

INSTITUTE OF EDUCATION SCIENCES 

NATIONAL CENTER FOR EDUCATION STATISTICS 

March, 2007 



The National Center for Education Statistics, part of the United States Department of Education, needs 
your help with an important education research study. The National Household Education Surveys Program 
(NHES) will be conducted in households all over the country to learn about the educational experiences of both 
adults and children — important issues we can only learn about by speaking with people like you. 

Your participation in this research study will help us learn about 

• The early educational experiences of young children and how families prepare for children to start 
school; 

• The ways that parents and families are involved in children’s education at school and at home, and 
how schools work with families; and 

• Types of education and training that adults may take part in. 

Your household was selected for the study as part of a scientific random sample of all households in your 
area, and another household cannot be substituted for yours. Your household represents thousands of other 
households. Even if there are no children or adults who have taken part in educational activities in your household, 
it is important that we talk to you so that the study results accurately reflect the experiences of all children and 
adults across the nation. 




Please be assured that all information you give is completely confidential and will never be published with 
your name. More details about the interviews, how your household was selected, and how to obtain reports from 
previous surveys are provided on the back of this letter. 



Westat, a social science research firm, will conduct this study. An interviewer with a Westat badge will 
visit your household sometime between March 18 and June 10, 2007. A few initial questions will determine if 
someone in your household is selected for an interview. If we happen to visit at an inconvenient time, please 
suggest a time that is better for you. If you would like to set an appointment or complete the interview by 
telephone before we visit, you can contact Westat at their toll-free number (1-888-696-5670) and give your 
telephone number and the time that is convenient for you. 



Please help us in our efforts to better understand education in the United States. We recognize that you 
have many demands on your time, and we thank you in advance for your cooperation in this important research. 



Sincerely, 



yt[. 



Gail Mulligan 
Project Officer 

National Household Education Surveys Program 



WASHINGTON, D.C. 20006 
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Some Frequently Asked Questions about the 
National Household Education Surveys Program (NHES) 



Q. How will the study results be used? What will you do with this information? 

A. The information we collect will be used to better understand educational experiences and needs. 

Findings will be published in U.S. Department of Education reports. Reports from NFIES surveys are 
available online at http://nces.ed.gov/nhes or by writing to the National Center for Education Statistics 
at the address below. The NFIES reports, which do not reveal individual answers but rather grouped 
data for large numbers of people, are widely distributed to educators, researchers, policy makers, news 
organizations, and the general public. 

Q. How did you select my household for the study? 

A. NFIES uses scientific sampling methods to select households for the study. Flouseholds were selected 
by drawing addresses from residential addresses in your area. Where possible we matched addresses 
with telephone numbers so that we can conduct as many interviews as possible by telephone, which is 
less expensive than in-person interviewing. 

Because we were unable to match your address with a phone number, an interviewer will be coming to 
visit your household between March 1 8 and June 1 0, 2007. 

Q. Will you keep my information confidential? 

A. All information you give to the interviewer will be kept completely confidential. Employees of the U.S. 
Department of Education and Westat who are working on this study are required by law to protect the 
confidentiality of respondents. Your responses will be protected to the fullest extent allowable under 
law. Also, individual responses are never published in reports; they are combined with the responses of 
others and are published as grouped data only. 

Q. How long will the survey take? 

A. First, there are a few short questions to see if any members of your household qualify for the study. 
They take about 5 minutes. In about half of all households, no one is selected for an interview. If 
someone is chosen for an interview, it will take approximately 15 to 30 minutes, on average, depending 
on the interview. 

Q. What is the authority for conducting this survey? 

A. This study has been approved by the Office of Management and Budget, the office that reviews all 
federally sponsored surveys. The approval number assigned to this study is 1850-0768. You may 
send any comments about this survey, including its length, to the Federal Government. Write to Gail 
Mulligan, National Center for Education Statistics, U.S. Department of Education, 1990 K Street NW, 
Room 9065, Washington, DC 20006-5650. You may send e-mail to nhes@ed.gov. 
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FA 




U.S. DEPARTMENT OF EDUCATION 

INSTITUTE OF EDUCATION SCIENCES 

NATIONAL CENTER FOR EDUCATION STATISTICS 



(Month), 2007 

Dear Sir or Madam: 

Recently, a professional interviewer visited your household for a research study about education. The 
National Household Education Surveys Program (NHES) is sponsored by the National Center for Education 
Statistics, part of the United States Department of Education. As of the date we mailed this letter, we had not 
completed an interview with your household. You may have already received a letter from me introducing 
NHES. I am writing to give you more information about the study. I hope that after reading it, you will take 
part in this important research effort. 

The purpose of this research is to learn about the educational experiences of both adults and children. 
We can only learn about these issues by speaking to families like yours. Your participation in the NHES will 
help us learn about 

• The early educational experiences of young children and how families prepare for children to start 
school; 

• The ways that parents and families are involved in children’s education at school and at home, and 
how schools work with families; and 

• Types of education and training that adults may take part in. 




Your household was selected for the study as part of a scientific random sample of households and 
another household cannot be substituted for yours. Your household represents thousands of other households. 
Even if no one in your household has taken part in educational activities, it is important that we talk to you so 
that the study results can reflect the experiences of all children and adults. Please be assured that the information 
you provide is completely confidential and will never be given out with your name. 

Westat, a social science research firm, is conducting this study. In the next week or two, an 
interviewer from Westat will visit your household again. If we happen to stop by at an inconvenient time, 
please suggest a time that is better for you. If you would like to set an appointment before we call, you can 
contact Westat at their toll-free number (1-888-696-5670) and give your telephone number and the time that 
is convenient for you. 

We know that you have many demands on your time, and we thank you in advance for your help in this 
important research effort to better understand education in the United States. 



Sincerely, 



yf[. 



Gail Mulligan 
Project Officer 

National Household Education Surveys Program 
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Some Frequently Asked Questions about the 
National Household Education Surveys Program (NHES) 



Q. How will the study results be used? What will you do with this information? 

A. The information we collect will be used to better understand educational experiences and needs. Findings 
will be published in U.S. Department of Education reports. Reports from NHES surveys are available 
online at http://nces.ed.gov/nhes or by writing to the National Center for Education Statistics at the 
address below. The NHES reports, which do not reveal individual answers but rather grouped data for 
large numbers of people, are widely distributed to educators, researchers, policy makers, news 
organizations, and the general public. 

Q. How did you select my household for the study? 

A. NHES uses scientific sampling methods to select households for the study. Households were selected by 
drawing addresses from residential addresses in your area. Where possible we matched addresses with 
telephone numbers so that we can conduct as many interviews as possible by telephone, which is less 
expensive than in-person interviewing. 

Because we were unable to match your address with a phone number, an interviewer will be coming 
to visit your household between March 18 and June 10, 2007. 

Q, Will you keep my information confidential? 

A. All information you give to the interviewer will be kept completely confidential. Employees of the U.S. 
Department of Education and Westat who are working on this study are required by law to protect the 
confidentiality of respondents. Your responses will be protected to the fullest extent allowable under law. 
Also, individual responses are never published in reports; they are combined with the responses of others 
and are published as grouped data only. 

Q, How long will the survey take? 

A. First, there are a few short questions to see if any members of your household qualify for the study. They 
take about 5 minutes. In about half of all households, no one is selected for an interview. If someone is 
chosen for an interview, it will take approximately 15 to 30 minutes, on average, depending on the 
interview. 

Q, What is the authority for conducting this survey? 

A. This study has been approved by the Office of Management and Budget, the office that reviews all 
federally sponsored surveys. The approval number assigned to this study is 1850-0768. You may send 
any comments about this survey, including its length, to the Federal Government. Write to Gail 
Mulligan, National Center for Education Statistics, U.S. Department of Education, 1990 K Street NW, 
Room 9065, Washington, DC 20006-5650. You may send e-mail to nhes@ed.gov. 

Q. I got a letter earlier. Why did I get another letter? 

A. Sometimes letters do not get delivered or are not read for a number of reasons. This letter is sent to 
assure that someone in your household has a chance to read information about the study. We are 
sending this by FedEx at the special low rate available to the government in order to bring this 
important study to your attention. 
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March, 2007 



Dear Sir or Madam: 

The bearer of this letter is a professionally trained interviewer working on the National Household 
Education Surveys Program (NHES) for the National Center for Education Statistics (NCES), which is 
part of the United States Department of Education. The interviewer is an employee of Westat, which was 
contracted by NCES to conduct this study. 

The interviewer is contacting designated households to gather important information about the 
educational experiences of adults and children. All households selected for this education research study 
have received a letter from the U.S. Department of Education explaining the project. The householder 
may decide if he or she wishes to participate. 

If you receive an inquiry from a concerned citizen, please assure that person of the legitimacy of the 
interviewer’s work. All interviewers working on this project have a photo identification badge. You may 
contact me at (202) 502-7491, or Westat at 1-888-696-5670, if you have any questions. 



Sincerely, 




Gail Mulligan 
Project Officer 

National Household Education Surveys Program 
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National Household Education Surveys Program (NHES) 

HOUSEHOLD FOLDER 



ADDRESS 






ADDRESS DISPOSITIONS 

(circle one) 

CA Correct address 

CF Correct with minor change 

UC ....Unable to confirm address 

NF Not Found 

NE No Entry 

NO Other (EXPLAIN) 







1. INTRODUCTION: 

Hello, I’m (NAME) with Westat, a social science research company located in Maryland. 
Here is my ID badge (SHOW BADGE). I am working on the National Household Education 
Surveys Program. This is a research study sponsored by the United States Department of 
Education to learn about educational experiences of both adults and children. 

(ASK IF NOT OBVIOUS; Are you a member of this household and at least 18 years old?) 

YES GIVE THE ADVANCE LETTER AND BROCHURE 

NO IF NO, ASK FOR HH MEMBER 18 OR OLDER (HEAD OF HH IF NON 18+) 

IF NO ADULT/HEAD OF HH AVAILABLE, SET APPOINTMENT AND END. 



2. First I need to verify that I am at the 
correct address. Is this (LABEL 
ADDRESS)? 

1. YES (GO TO Q3) 

2. YES with minor change (GO TO q3) 

3. NO ADDRESS MATCH (GO TO 05) 



3. Is (LABEL TELEPHONE NUMBER) a 
telephone number in this household? 

1 . YES (GO TO q4) 

2. NO (GO TO Q4) 
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4. CORRECT ADDRESS LOCATED: 

As I said, this research study is sponsored by the United States Department of 
Education, and is concerned with the education of both children and adults. 
Interviews are being conducted by our professionally trained interviewing staff in 
Rockville, MD. We can call them toll-free on your phone or use this cell phone. 

First, a few questions about the people in your household will identify anyone who is 
eligible for an interview. We are offering $20 to those households that complete this 
initial screening interview. After those initial questions are done, you or another 
member of your household may be selected for an interview. 

(GO TO PAGE 3) 



5. IF ADDRESS IS NOT A MATCH: 

THANK RESPONDENT: 

Thank you, that’s all the information that I need. We have the wrong address so we 
will not be asking you for an interview. 



SETTING AN APPOINTMENT 

IF HH MEMBER AT LEAST 18 YEARS OLD IS NOT AVAILABLE OR IF 
RESPONDENT CANNOT DO THE INTERVIEW NOW, SET APPOINTMENT 
TO RETURN. 

When would be a good time for me to return so that we can conduct the 
interview with (you/NAME)? 

APPOINTMENT WITH: 

DATE: TIME: AM/PM 

LEAVE APPOINTMENT CARD WITH HOUSEHOLD 



NOTE: REMEMBER TO MAKE CALL RECORD ENTRIES ON PAGE 4 AND GIVE ADDRESS LABEL A 
DISPOSITION CODE. 

IF ADDRESS IS CORRECT, MAKE TELEPHONE CORRECTIONS ON LABEL, IF APPLICABLE. 
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MAKING CONTACT WITH THE TRC 

IF USING THE HOUSEHOLD’S TELEPHONE - I will place the call to 1-888-696- 
5672, which is a toll free number at the interviewing center. When the supervisor 
answers, I will give her some information (I will give the supervisor your telephone 
number in case we are disconnected. [GAIN PHONE NUMBER IF IT IS NOT LISTED 
ON THE LABEL]). After that, I will hand the telephone to you and the interviewer will 
ask you a few questions about the people in your household. Once that is completed 
the interviewer will ask to speak to me once more. I will give the phone back to you or 
let you know if another member of your household is selected. Then I will leave and 
you may continue the interview in private. 

IF USING THE CELLULAR TELEPHONE - To ensure good telephone reception, let’s 
sit close to a window or outside. I will place the call to 1-888-696-5672, which is a toll 
free number at the interviewing center. When the supervisor answers, I will give her 
some information. After that, I will hand the telephone to you and the interviewer will 
ask you a few questions about the people in your household. If the call becomes 
disconnected or if static or other interference interrupts the interview, please hand the 
telephone back to me. Please do not turn off the power or push any buttons during 
the interview. At the end of the initial questions about your household, the interviewer 
will ask to speak to me again. After that, I will hand the telephone back to you so that 
you may continue the interview, or let you know if another household member is 
chosen. 

INTERVIEWER INSTRUCTIONS : WHEN THE TRC SUPERVISOR ANSWERS, 
PROVIDE HER WITH YOUR NAME AND THE IDENTIFICATION NUMBER ON THE 
LABEL. REMIND TRC INTERVIEWER TO ASK FOR YOU ONCE THE SCREENER 
IS COMPLETED. ONCE THE CALL HAS BEEN TRANSFERRED TO A TRC 
INTERVIEWER, HAND THE TELEPHONE TO THE RESPONDENT. 

SPANISH SPEAKING HOUSEHOLD : IF YOU ENCOUNTER A HOUSEHOLD WHERE 
SPANISH IS THE ONLY LANGUAGE SPOKEN, AND YOU ARE NOT A SPANISH 
SPEAKING INTERVIEWER, ASK FOR AN ENGLISH-SPEAKING HOUSEHOLD 
MEMBER. IF AN ENGLISH-SPEAKING HOUSEHOLD MEMBER, AGE 18 YEARS OR 
OLDER IS AVAILABLE, PROCEED WITH INTERVIEW. IF NO ENGLISH SPEAKING 
HOUSEHOLD MEMBER IS AVAILABLE, AND YOU ARE UNABLE TO 
COMMUNICATE WITH THE HOUSEHOLD, TRY TO CONFIRM WITH SOMEONE IN 
THE HOUSEHOLD THAT THE ADDRESS (AND TELEPHONE NUMBER, IF ONE IS 
PRESENT) PRINTED ON THE LABEL IS CORRECT. THANK THE RESPONDENT 
AND THEN CONTACT YOUR SUPERVISOR. 
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CALL RECORD 



DATE 


DAY OF 
WEEK 


TIME 


RESULT 


HOME/CELL 

(CIRCLE ONE ON 
DATE CASE IS 
FINALIZED) 


COMMENTS 


DU 

TYPE 






AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 







DU STRUCTURE TYPE 


INTERIM CODES (for 


FINAL CODES 


SA StancJ alone home 


Screeners onlv) 

1 Not home/no answer 


01 Complete, screener, no 

eligibles 

02 Complete, field screener and 


TH Town home/Duplex 


2 Refusal/breakoff 

4 Appointment 


extended complete 
03 Complete, field screener, 1 or 


AP Apartment, Flat 


6 Language problem 


more extended interview 
outstanding 


OS ...Other 


7 No entry 

8 Other 


RB Refusal/breakoff 

MC Maximum calls 

LP Language Problem 

NE No Entry 

NF Not Found 

NV 

Vacant/demolished/condemne 

d 

ND Not a DU 

NO Other 
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Programa Nacional de Encuestas en Hogares sobre Educacion (NHES) 

HOUSEHOLD FOLDER 



ADDRESS 






ADDRESS DISPOSITIONS 

(circle one) 

CA .... Correct address 

CF Correct with minor change 

UC ... Unable to confirm address 

NF Not Found 

NE .... No Entry 
NO Other (EXPLAIN) 







1. INTRODUCTION: 

Buenos dias/Buenas tardes, mi nombre es (NAME) y trabajo para Westat, una firma 
localizada en Maryland que realiza investigacion en el area de las ciencias sociales. 
Esta es mi identificacion (SHOW BADGE). Estoy trabajando para el Programa 
Nacional de Encuestas en Hogares sobre Educacion. Este es un estudio patrocinado 
por el Departamento de Educacion de los Estados Unidos para recoger informacion 
acerca de las experiencias educativas de adultos y nines. (ASK IF NOT OBVIOUS; 
l,Es usted miembro de este hogar y tiene por lo menos 18 anos de edad?) 

YES GIVE THE ADVANCE LETTER AND BROCHURE 

NO IF NO, ASK FOR HH MEMBER 18 OR OLDER (HEAD OF HH IF NON 18+) 

IF NO ADULT/HEAD OF HH AVAILABLE, SET APPOINTMENT AND END. 



2. Primero, necesito verificar que estoy en 
la direccion correcta. <;,Es esta 
(LABEL ADDRESS)? 

1. YES (GO TO Q3) 

2. YES with minor change (GO TO q3) 

3. NO ADDRESS MATCH (GO TO 05) 



3. <i,Es (LABEL TELEPHONE NUMBER) 
un numero de telefono en esta casa? 

3. YES (GO TO q4) 

4. NO (GO TO q4) 
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4. CORRECT ADDRESS LOCATED: 

Tal como mencione, este estudio de investigacion es patrocinado por el 
Departamento de Educacion de los Estados Unidos, y se trata sobre la educacion de 
nines y adultos. Las entrevistas son realizadas por nuestro personal entrenado 
profesionalmente en Rockville, MD. Podemos llamarlos gratis en su telefono o usar 
este celular. 

Primero, hay unas preguntas acerca de las personas en su hogar para ver si hay 
alguien que cumple con los requisites para una entrevista. Estamos ofreciendo $20 a 
los hogares que completan esta entrevista inicial. Despues que terminemos con esas 
preguntas, usted u otro miembro de su hogar puede ser elegido para ser entrevistado. 

(GO TO PAGE 3) 



5. IF ADDRESS IS NOT A MATCH: 

THANK RESPONDENT: 

Muchas gracias, esa es toda la informacion que necesito. La direccion que tenemos 
no es correcta, asi es que no le pedire una entrevista. 



SETTING AN APPOINTMENT 

IF HH MEMBER AT LEAST 18 YEARS OLD IS NOT AVAILABLE OR IF 
RESPONDENT CANNOT DO THE INTERVIEW NOW, SET APPOINTMENT 
TO RETURN. 

<i,Cuando seria conveniente que yo regresara para hacer la entrevista con 
(usted/NAME)? 

APPOINTMENT WITH: 

DATE: TIME: AM/PM 

LEAVE APPOINTMENT CARD WITH HOUSEHOLD 



NOTE: REMEMBER TO MAKE CALL RECORD ENTRIES ON PAGE 4 AND GIVE ADDRESS LABEL A 
DISPOSITION CODE. 

IF ADDRESS IS CORRECT, MAKE TELEPHONE CORRECTIONS ON LABEL, IF APPLICABLE. 
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MAKING CONTACT WITH THE TRC 

IF USING THE HOUSEHOLD’S TELEPHONE - Voy a marcar el 1-888-696-5672. 

Este numero es del centre de entrevistas y es una llamada gratis. Cuando la 
supervisora conteste, le dare alguna informacion (le dare a la supervisora su numero 
de telefono en case que la llamada se desconecte. [GAIN PHONE NUMBER IF IT IS 
NOT LISTED ON THE LABEL]). Despues, le voy a pasar el telefono a usted y el 
entrevistador le hara algunas preguntas sobre las personas de su hogar. Una vez que 
termine, el entrevistador pedira hablar conmigo de nuevo. Despues le voy a pasar a 
usted el telefono o le voy a decir si otro miembro de su hogar ha sido elegido. En ese 
momento me voy a ir para que usted pueda continuar la entrevista en privado. 

IF USING THE CELLULAR TELEPHONE - Para asegurarnos que hay buena 
recepcion, sentemonos cerca de una ventana o afuera. Voy a marcar el 1-888-696- 
5672. Este numero es del centre de entrevistas y es una llamada gratis. Cuando la 
supervisora conteste, le dare alguna informacion. Despues, le voy a pasar el telefono 
a usted y el entrevistador le hara algunas preguntas sobre las personas de su hogar. 

Si la llamada se desconecta o si la entrevista se interrumpe con estatica u otra 
interferencia, por favor paseme el telefono. Por favor no lo apague ni presione ningun 
boton durante la entrevista. Una vez que termine las preguntas iniciales sobre su 
hogar, el entrevistador pedira hablar conmigo de nuevo. Despues le voy a pasar a 
usted el telefono para que pueda continuar con la entrevista, o le voy a decir si otro 
miembro de su hogar ha sido elegido. 

INTERVIEWER INSTRUCTIONS : WHEN THE TRC SUPERVISOR ANSWERS, 
PROVIDE HER WITH YOUR NAME AND THE IDENTIFICATION NUMBER ON THE 
LABEL. REMIND TRC INTERVIEWER TO ASK FOR YOU ONCE THE SCREENER 
IS COMPLETED. ONCE THE CALL HAS BEEN TRANSFERRED TO A TRC 
INTERVIEWER, HAND THE TELEPHONE TO THE RESPONDENT. 

SPANISH SPEAKING HOUSEHOLD : IF YOU ENCOUNTER A HOUSEHOLD WHERE 
SPANISH IS THE ONLY LANGUAGE SPOKEN, AND YOU ARE NOT A SPANISH 
SPEAKING INTERVIEWER, ASK FOR AN ENGLISH-SPEAKING HOUSEHOLD 
MEMBER. IF AN ENGLISH-SPEAKING HOUSEHOLD MEMBER, AGE 18 YEARS OR 
OLDER IS AVAILABLE, PROCEED WITH INTERVIEW. IF NO ENGLISH SPEAKING 
HOUSEHOLD MEMBER IS AVAILABLE, AND YOU ARE UNABLE TO 
COMMUNICATE WITH THE HOUSEHOLD, TRY TO CONFIRM WITH SOMEONE IN 
THE HOUSEHOLD THAT THE ADDRESS (AND TELEPHONE NUMBER, IF ONE IS 
PRESENT) PRINTED ON THE LABEL IS CORRECT. THANK THE RESPONDENT 
AND THEN CONTACT YOUR SUPERVISOR. 
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CALL RECORD 



DATE 


DAY OF 
WEEK 


TIME 


RESULT 


HOME/CELL 

(CIRCLE ONE ON 
DATE CASE IS 
FINALIZED) 


COMMENTS 


DU 

TYPE 






AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 










AM 

PM 




H C 







DU STRUCTURE TYPE 


INTERIM CODES (for 


FINAL CODES 


SA StancJ alone home 


Screeners only) 

1 Not home/no answer 


01 Complete, screener, no 

eligibles 

02 Complete, field screener and 


TH Town home/Duplex 


2 Refusal/breakoff 

4 Appointment 


extended complete 
03 Complete, field screener, 1 or 


AP Apartment, Flat 


6 Language problem 


more extended interview 
outstanding 


OS ....Other 


7 No entry 

8 Other 


RB .... Refusal/breakoff 
MC.... Maximum calls 

LP Language Problem 

NE .... No Entry 

NF Not Found 

NV 

Vacant/demolished/condemne 

d 

ND .... Not a DU 
NO .... Other 
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This page is intentionally blank. 




NHES:2007 INTERVIEWER OBSERVATION FORM 



FIELDID: (print or affix label) Interviewer initials: 

Record date and time lOF was completed: / / : am/pm 



Neighborhood Characteristics 

IOF1100. How would you characterize the area? 



URBANCH URBAN 1 

SUBURBAN 2 

RURAL 3 

IOF1 120. What is the traffic flow on this street? 

TRAFFICF LIGHT 1 

MODERATE 2 

HEAVY 3 

IOF1 140. Are there abandoned cars on the street or in the alley or lots nearby? 

ABANDCAR NONE 1 

VERY FEW (ROUGHLY 1-2 CARS) 2 

SOME (ROUGHLY 3-4 CARS) 3 

MANY (ROUGHLY 5 OR MORE CARS) 4 



IOF1 160. Is there trash or junk on the street or sidewalks, in yards or lots? 



TRASHVIS NONE 1 

VERY LITTLE 2 

SOME 3 

A LOT 4 

IOF1180. How would you characterize the land use on this block? 

LANDUSE PRIMARILY RESIDENTIAL (HOUSES AND 

APARTMENTS) 1 

PRIMARILY COMMERCIAL (STORES AND 

BUSINESSES) 2 

PRIMARILY INDUSTRIAL (WAREHOUSES AND 

FACTORIES) 3 

PRIMARILY VACANT LOTS OR UNDEVELOPED 

SPACE 4 

MIXED RESIDENTIAL AND COMMERCIAL 5 

MIXED RESIDENTIAL AND INDUSTRIAL 6 

MIXED RESIDENTIAL AND VACANT LOTS 7 

OTHER 91 

SPECIFY 
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IOF1200. 



How many houses/apartments have window bars or gratings on doors or windows? 



SECURBAR NONE 1 

VERY FEW 2 

SOME 3 

MOST 4 

ALL 5 



IOF1220. How many houses/apartments have signs indicating they are protected by private security services? 



SECURSVC NONE 1 

VERY FEW 2 

SOME 3 

MOST 4 

ALL 5 



IOF1240. Are there signs indicating there is a neighborhood watch on this block? 



SECURNWT YES 1 

NO 2 

IOF1260. Did you see any children (infants/toddlers/children/teenagers) on the block? 

CHILDBLK YES 1 

NO 2 

IOF1280. Did you hear any language other than English on the block? 

OTHLANG YES 1 

NO 2 

NO PEOPLE AROUND OR DID NOT HEAR ANY 
LANGUAGE 3 



Household Environment 



IOF1400. How would you characterize this household based on the home and exterior appearance? 



WEALTHHH AFFLUENT OR UPPER MIDDLE CLASS 1 

MIDDLE-MIDDLE CLASS 2 

WORKING CLASS OR POOR 3 



IOF1420. Is there any evidence of children living in this household? [Look for playsets in the yard, children’s 
bicycles or toys, children playing in the yard, etc.] 



CHILDHH YES 1 

NO 2 



IOF1440. Does this household have window bars or gratings on doors or windows? 



SECBARHH YES 1 

NO 2 



IOF1460. Does this household have signs indicating they are protected by private security services? 



SECSVCHH YES 1 

NO 2 
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Sorry I Missed You! 




National Household 

Education Surveys Program 



Sponsored by: 

National Center for Education Statistics 
United States Department of Education 
nces.ed.gov/nhes 



Sorry I Missed You! 

I stopped by your home today to talk with you about the National Household 
Education Surveys Program (NHES). This is an important education research 
study sponsored by the United States Department of Education and conducted 
by Westat, a social science research organization. 

We are not selling anything! 

I will return to speak with you in a few days. If you would like more information or 
would like to set up an appointment for a conf idential interview please contact 

Westat toll-free at 888-696-5670. 

Thank you in advance for your participation! 



OMB# 1850-0768 
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iSiento no haberle encontrado! 




National Household 

Education Surveys Program 



Patrocinado por: 

Centro Nacional de Estadisticas de Educacion 
Departamento de Educacion de Estados Unidos 
nces.ed.gov/nhes 



iSiento no haberle encontrado! 

Pase hoy por su casa para hablar con usted sobre el Programa Nacional de 
Encuestas en Hogares sobre Educacion (NHES). Este es un importante estudio 
de investigacion sobre educacion patrocinado por el Departamento de 
Educacion de Estados Unidos y conducido 
por Westat, una firma de investigacion en el area de las ciencias sociales. 

iNo vendemos nada! 



Regresare para hablar con usted dentro de unos dias. Si desea mds informacion 
0 quisiera programar una cita para hacer una entrevista confidencial, por favor 
Name a Westat al numero gratuito 888-696-5670. 

iDe antemano, muchas gracias por su participacidn! 

OMB# 1850-076 
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National Household 

Education Surveys Program 



Sponsored by: 

National Center for Education Statistics 
United States Department of Education 

nces.ed.gov/nhes 
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Appointment Card 

I stopped by your home today to talk with you about the National Household 
Education Surveys Program (NHES). This is an important education research 
study sponsored by the United States Department of Education and conducted 
by Westat, a social science research organization. 

We are not selling anything! 

I will return on: 

At to speak with 

If it's necessary to change your appointment for a conf idential interview or if 
you would like additional information, please contact Westat at 1-888-696-5670. 

Or you may contact me at: 



Thank you in advance for your participation! 

0MB# 1850-0768 
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Field ID 



National Household Education Surveys Program (NHES) 

Follow-Up Study 

NON-INTERVIEW REPORT FORM (NIRF) 



Address: 



City, State, ZIP: 



1, To whom did you speak to during your contact attempts: 



Name: 


Name: 


Name: 


Age: 


Age: 


Age: 


Sex: 


Sex: 


Sex: 


Race: 


Race: 


Race: 


2. 


Why were you unahle to get a completion? 





Language barrier or problem 1(Q5) 



Refusal/Breakoff 2(Q3) 

Maximum Calls 3(Q6) 

Not Found 4(Q6) 

No entry 5(Q6) 

Other (specify) 6(Q6) 
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3 . 



IF REFUSAL/BREAKOFF: What reasons were given for the 
refusal/hreakoff? (RECORD EXPLANATION “R” GAVE.] 



4. 

CODE THE MOST IMPORTANT REASON FOR THE RESPONDENT’S 
REFUSAL/BREAKOFF, CIRCLE ONLY ONE 

Too Busy 1 

Too 111 SP/Family Member 2 

Not Interested/Didn’t want to Partieipate 3 

Doesn’t like Survey/Questions 4 

Dislikes Government/Waste of Time 5 

Afraid/Told not to Partieipate/Gatekeeper 6 

Confidentiality/No Consent/Voluntary 7 

Claimed this Survey did not Apply to Respondent 8 

Other Speeify 



GO TO Q6. 



5, IF LANGUAGE PROBLEM: Record the language spoken in household. 



6. DESCRIBE THE CIRCUMSTANCES OF THE NON-RESPONSE 



7. Interviewer: 
Date: 



F-2 




Appendix G 



Nonresponse Bias Estimates With Unadjusted Base 

Weights 




This page is intentionally blank. 




Table G-1. Unadjusted base weight estimates of nonresponse bias for various characteristics from the School Readiness (SR) Survey: 
NHES:2007 Bias Study 





Reduced treatment SR respondents 


Full treatment SR respondents 


Bias' 




Relative bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Participation in center-based care 


143 


54.6 


3.34 


177 


57.1 


3.22 


-2.5 


1.08 


-4.4 


Specific skills 


Recognizes all colors 


207 


84.6 


1.61 


249 


85.4 


1.55 


-0.8 


0.57 


-0.9 


Counts to 20 or higher 


165 


66.9 


2.73 


198 


67.7 


2.42 


-0.9 


0.77 


-1.2 


Recognizes all letters 


78 


30.0 


2.33 


93 


31.0 


1.87 


-1.0 


1.00 


-3.2 


Writes first name 


158 


63.1 


3.41 


193 


65.0 


3.05 


-1.9 


0.84 


-2.9 


Holds a pencil 
Speech is often 


210 


86.9 


2.33 


252 


87.3 


1.67 


-0.5 


0.82 


-0.5 


understandable to a stranger 
Reads or pretends to read 


207 


85.7 


2.36 


249 


86.4 


1.97 


-0.6 


0.60 


-0.8 


storybooks 

Parents believe it is essential to do 
certain things to prepare child 
for kindergarten 


242 


98.9 


0.69 


289 


99.0 


0.61 


-0.2 


0.09 


-0.1 


Teach child the alphabet 


143 


57.1 


2.62 


171 


57.6 


2.40 


-0.4 


1.22 


-0.9 


Teach child about sharing 


149 


59.4 


2.26 


177 


59.6 


2.44 


-0.1 


0.95 


-0.3 


Teach child to read 


117 


48.6 


2.78 


139 


48.4 


2.16 


0.2 


1.45 


0.4 


Teach child numbers 
Show child how to hold a 


129 


51.4 


2.46 


155 


52.1 


2.13 


-0.7 


1.21 


-1.3 


pencil 


99 


38.3 


3.11 


121 


39.5 


2.44 


-1.2 


1.14 


-3.0 


Family member read to child 


everyday in the past week 


145 


58.0 


3.31 


169 


57.3 


2.75 


0.7 


0.82 


1.2 



See notes at end of table. 
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Table G-1. Unadjusted base weight estimates of nonresponse bias for various characteristics from the School Readiness (SR) Survey: 
NHES:2007 Bias Study — Continued 



Reduced treatment SR respondents Full treatment SR respondents Bias* Relative bias^ 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Parents report usually doing certain 
reading-related activity with 
child 

Ask child what is in a picture 


133 


54.6 


3.17 


159 


54.8 


3.15 


-0.2 


1.06 


-0.4 


Stop reading and point out 
letters 


81 


32.8 


3.01 


95 


32.5 


2.95 


0.3 


1.04 


0.9 


Ask child to read with parent 


72 


28.7 


2.79 


84 


28.3 


2.28 


0.5 


1.13 


1.4 


Talk about the story and what 
happened 


140 


58.9 


3.42 


165 


58.3 


2.81 


0.7 


1.13 


1.0 


Parents did home activities with 
child in the past week^ 


104 


41.1 


3.32 


123 


41.2 


3.05 


0.0 


1.22 


-0.2 


Parents took 3 or more outings with 
child in the past month"* 


115 


48.9 


3.15 


138 


48.9 


2.98 


0.0 


1.21 


0.0 


Child watches 2 or more hours of 
TV in a typical weekday 


151 


68.2 


2.62 


179 


67.1 


2.52 


1.1 


1.27 


1.6 


Child has a disability 


30 


13.7 


2.85 


37 


13.6 


2.51 


0.1 


0.67 


0.7 


Child’s age 
3 years 


108 


42.9 


3.44 


128 


42.2 


3.54 


0.7 


0.99 


1.7 


4 years 


95 


37.1 


3.47 


116 


38.5 


3.45 


-1.3 


1.16 


-3.6 


5 years and older 


42 


20.0 


4.46 


48 


19.4 


3.94 


0.6 


1.06 


3.1 


Child’s sex 
Male 


145 


60.0 


4.20 


169 


58.3 


3.35 


1.7 


1.39 


2.9 


Female 


100 


40.0 


4.20 


123 


41.7 


3.35 


-1.7 


1.39 


-4.1 



See notes at end of table. 
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Table G-1. Unadjusted base weight estimates of nonresponse bias for various characteristics from the School Readiness (SR) Survey: 
NHES:2007 Bias Study — Continued 



Reduced treatment SR respondents Full treatment SR respondents Bias* Relative bias^ 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Flousehold urbanicity 


Urban 


205 


83.1 


5.89 


247 


83.9 


5.41 


-0.7 


0.74 


-1.0 


Rural 


40 


16.9 


5.89 


45 


16.1 


5.41 


0.7 


0.74 


5.0 


Flome tenure 


Own 


165 


64.3 


3.30 


201 


65.5 


2.92 


-1.2 


0.86 


-1.8 


Rent/other 


80 


35.7 


3.30 


91 


34.5 


2.92 


1.2 


0.86 


3.5 


Parents’ educational attainment 


Fligh school diploma or below 


76 


34.9 


2.80 


91 


34.2 


2.81 


0.6 


1.10 


2.0 


Beyond high school diploma 


169 


65.1 


2.80 


201 


65.8 


2.81 


-0.6 


1.10 


-1.1 


Parents’ language 

Both/only parent(s) speak(s) 


English 


212 


86.6 


3.28 


254 


87.6 


2.92 


-1.0 


0.58 


-1.1 


One of two parents speaks 


English 


t 


t 


t 


t 


t 


t 


t 


t 


t 


No parent speaks English 


26 


12.5 


3.28 


29 


11.7 


2.92 


0.9 


0.57 


6.8 


Mothers’ employment status 


35 hours or more per week 


89 


37.4 


2.52 


111 


39.2 


2.54 


-1.8 


0.66 


-4.6 


Less than 35 hours per week 


55 


20.9 


3.15 


69 


21.6 


2.64 


-0.7 


0.85 


-3.2 


Looking for work 


23 


11.4 


2.53 


23 


9.9 


2.24 


1.5 


0.35 


15.2 


Not in labor force 


74 


28.6 


3.01 


83 


27.0 


2.67 


1.5 


0.47 


5.9 


No mother in household 


4! 


1.7! 


1.13! 


6 


2.2 


0.94 


-0.5 


0.63 


-22.7! 



See notes at end of table. 
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Table G-1. Unadjusted base weight estimates of nonresponse bias for various characteristics from the School Readiness (SR) Survey: 
NHES:2007 Bias Study — Continued 





Reduced treatment SR respondents 


Full treatment SR respondents 


Bias' 




Relative bias^ 


Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Poverty status 


Poor 


61 


28.6 


3.86 


80 


31.0 


3.26 


-2.4 


1.24 


-7.7 


Nonpoor 


184 


71.4 


3.86 


212 


69.0 


3.26 


2.4 


1.24 


3.5 


Household income 


Less than $15,000 


39 


18.0 


3.11 


52 


20.1 


2.60 


-2.1 


1.08 


-10.4 


$15,001 to $30,000 


47 


20.9 


2.86 


62 


22.1 


2.68 


-1.2 


1.12 


-5.4 


$30,001 to $50,000 


42 


16.0 


2.85 


50 


15.9 


2.41 


0.1 


0.87 


0.6 


More than $50,000 


117 


45.1 


2.89 


128 


41.9 


2.54 


3.2 


1.40 


7.6 


Family structure 


Mother and father 


187 


74.6 


5.20 


219 


73.2 


4.48 


1.4 


1.42 


1.9 


Mother 


49 


21.1 


4.10 


60 


21.6 


3.30 


-0.4 


1.42 


-2.3 


Father 


4! 


1.7! 


1.13! 


6 


2.2 


0.94 


-0.5 


0.63 


-22.7! 


Nonparent guardian(s) 


5 


2.6 


1.20 


7 


3.0 


1.28 


-0.4 


0.51 


-13.3 



# Rounds to zero. 

! Interpret with caution. 

X Reporting standards not met. 

' Bias is estimated as the difference between the reduced treatment SR respondents estimate and the full treatment SR respondents estimate. 

^ Relative bias is estimated as the bias estimate divided by the full treatment SR respondents estimate, and is expressed as a percentage. 

^ Told child a story; taught child letters, words, or numbers; taught child songs or music; did arts and crafts with child; played sports, active games or exercised 
together; and played board games or did puzzles with child. 

^ Any 3 or more of the following: Visited a library; visited a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or historical 
site; visited a zoo or aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting event (outside of 
school) in which the child was not a player. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, School Readiness (SR) Survey of the National Household Education Surveys 
Program, 2007. 
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Table G-2. Unadjusted base weight estimates of nonresponse bias for various characteristics from the Parent and Family Involvement in 
Education (PFI) Survey: NHES:2007 Bias Study 





Redueed treatment PFI respondents 


Full treatment PFI respondents 


Bias' 




Relative bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Parents partieipate in 5 or more 
aetivities in the student’s 
sehooP 


478 


49.7 


1.74 


566 


50.5 


1.59 


-0.8 


0.57 


-1.6 


Parents report sehool provides 
information very well 
About how student is doing in 
sehool 


568 


59.3 


2.48 


668 


59.5 


2.21 


-0.3 


0.51 


-0.3 


About how to help student 
with his/her homework 


449 


48.5 


1.65 


525 


48.3 


1.60 


0.2 


0.53 


0.4 


About why student is plaeed in 
partieular groups or elasses 


419 


45.1 


1.89 


496 


45.6 


1.64 


-0.5 


0.55 


-1.1 


About how to help student plan 
for eollege or voeational 
sehool 


205 


38.4 


1.61 


237 


37.6 


1.75 


0.8 


0.35 


2.1 


About the family’s expeeted role 
at student’s sehool 


460 


48.2 


1.68 


539 


48.2 


1.55 


0.1 


0.41 


0.0 


Parent reports being very satisfied 
with 4 or more aspeets of the 
student’s sehool"* 


685 


74.1 


1.40 


798 


73.6 


1.32 


0.5 


0.45 


0.7 


Parents partieipated in 5 or more 
home learning aetivities^ 


194 


43.3 


2.17 


229 


43.2 


2.01 


0.1 


0.55 


0.2 


Parents took 3 or more outings with 
student in the past month^ 


472 


50.3 


2.16 


559 


51.0 


1.96 


-0.7 


0.45 


-1.4 



See notes at end of table. 
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Table G-2. Unadjusted base weight estimates of nonresponse bias for various characteristics from the Parent and Family Involvement in 
Education (PFI) Survey: NHES:2007 Bias Study — Continued 





Redueed treatment PFI respondents 


Full treatment PFI respondents 


Bias' 




Relative bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Parents eheek to see that student’s 
homework gets done 


748 


86.8 


1.52 


885 


87.3 


1.34 


-0.5 


0.36 


-0.6 


Parents reeeived information about 
free tutoring 


396 


42.0 


2.25 


465 


42.4 


2.24 


-0.4 


0.71 


-0.9 


Parent expeets student to earn a 
eollege degree or higher 


369 


68.4 


3.02 


424 


67.8 


2.43 


0.6 


0.93 


0.9 


Family plans to help pay for 
student’s edueation after high 
sehool 


414 


83.1 


1.91 


475 


82.4 


1.73 


0.7 


0.68 


0.8 


Student partieipated in sehool 
aetivities 


531 


56.2 


2.70 


624 


56.3 


2.42 


-0.1 


0.58 


-0.2 


Student has a disability 


211 


22.4 


1.86 


257 


23.4 


1.80 


-1.0 


0.53 


-4.3 


Student’s sex 
Male 


500 


51.6 


2.00 


575 


50.8 


1.90 


0.8 


0.39 


1.6 


Female 


459 


48.4 


2.00 


548 


49.2 


1.90 


-0.8 


0.39 


-1.6 


Flousehold urbanieity 
Urban 


797 


84.0 


4.61 


933 


84.1 


4.79 


0.0 


0.37 


-0.1 


Rural 


162 


16.0 


4.61 


190 


15.9 


4.79 


0.0 


0.37 


0.6 


Flome tenure 
Own 


708 


71.5 


1.17 


837 


72.4 


0.97 


-0.9 


0.59 


-1.2 


Rent/other 


251 


28.5 


1.17 


286 


27.6 


0.97 


0.9 


0.59 


3.3 



See notes at end of table. 
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Table G-2. Unadjusted base weight estimates of nonresponse bias for various characteristics from the Parent and Family Involvement in 
Education (PFI) Survey: NHES:2007 Bias Study — Continued 



Reduced treatment PFI respondents Full treatment PFI respondents Bias' Relative bias^ 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Parents’ educational attainment 


High school diploma or below 


269 


30.2 


2.47 


326 


30.8 


2.27 


-0.5 


0.54 


-1.9 


Beyond high school diploma 


690 


69.8 


2.47 


797 


69.2 


2.27 


0.5 


0.54 


0.9 


Parents’ language 

Both/only parent(s) speak(s) 


English 


861 


90.5 


2.19 


1012 


91.0 


2.17 


-0.5 


0.16 


-0.5 


One of two parents speaks 


English 


20 


2.1 


0.47 


22 


2.0 


0.51 


0.1 


0.10 


5.0 


No parent speaks English 


59 


7.4 


2.01 


66 


7.0 


1.94 


0.4 


0.15 


5.7 


Mothers’ employment status 


35 hours or more per week 


445 


45.1 


2.61 


521 


45.2 


2.38 


0.0 


0.52 


-0.2 


Less than 35 hours per week 


219 


22.2 


2.15 


260 


22.5 


1.84 


-0.3 


0.58 


-1.3 


Looking for work 


41 


4.7 


0.75 


45 


4.5 


0.61 


0.2 


0.24 


4.4 


Not in labor force 


207 


23.9 


1.67 


241 


23.8 


1.29 


0.2 


0.67 


0.4 


No mother in household 


47 


4.0 


0.55 


56 


4.0 


0.56 


-0.1 


0.18 


# 


Poverty status 


Poor 


164 


21.9 


2.51 


221 


23.8 


2.41 


-1.9 


0.47 


-8.0 


Nonpoor 


795 


78.1 


2.51 


902 


76.2 


2.41 


1.9 


0.47 


2.5 


Household income 


Less than $15,000 


115 


14.8 


1.68 


153 


15.6 


1.45 


-0.8 


0.42 


-5.1 


$15,001 to $30,000 


141 


15.2 


1.66 


179 


16.5 


1.72 


-1.3 


0.58 


-7.9 


$30,001 to $50,000 


152 


15.8 


1.75 


171 


15.2 


1.63 


0.6 


0.24 


3.9 


More than $50,000 


551 


54.2 


2.64 


620 


52.7 


2.54 


1.6 


0.49 


2.8 



See notes at end of table. 
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Table G-2. Unadjusted base weight estimates of nonresponse bias for various characteristics from the Parent and Family Involvement in 
Education (PFI) Survey: NHES:2007 Bias Study — Continued 



Reduced treatment PFI respondents Full treatment PFI respondents Bias' Relative bias^ 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Family structure 


Mother and father 


706 


74.9 


1.66 


823 


74.8 


1.64 


# 


0.39 


0.1 


Mother 


172 


18.3 


1.32 


204 


18.3 


1.17 


0.1 


0.41 


# 


Father 


45 


3.7 


0.48 


54 


3.8 


0.49 


-0.1 


0.18 


-2.6 


Nonparent guardian(s) 


36 


3.1 


0.49 


42 


3.1 


0.52 


# 


0.16 


# 


School type 


Public 


840 


89.5 


1.27 


989 


89.7 


1.28 


-0.2 


0.24 


-0.2 


Private 


101 


10.5 


1.27 


114 


10.3 


1.28 


0.2 


0.24 


1.9 


School size 


Under 300 


148 


16.3 


1.44 


167 


15.5 


1.32 


0.8 


0.29 


5.2 


300-599 


275 


30.2 


2.75 


321 


30.0 


2.68 


0.2 


0.26 


0.7 


600-999 


241 


25.5 


2.15 


286 


25.9 


2.06 


-0.5 


0.51 


-1.5 


1 ,000 or more 


272 


28.0 


2.30 


323 


28.5 


2.03 


-0.5 


0.54 


-1.8 



# Rounds to zero. 

' Bias is estimated as the difference between the reduced treatment PFI respondents estimate and the full treatment PFI respondents estimate. 

^ Relative bias is estimated as the bias estimate divided by the full treatment PFI respondents estimate, and is expressed as a percentage. 

^ Any 5 or more of the following: Attended a general school meeting; attended a meeting of the parent-teacher organization or association; went to a regularly 
scheduled parent-teacher conference with the student’s teacher; attended a school or class event because of the student; served as a volunteer in the student’s 
classroom or elsewhere in the school; participated in fundraising for the school; served on a school committee; and met with a guidance counselor in person. 

Any 4 or more of the following: School student attends this year; teachers student has this year; academic standards of the school; order and discipline at the 
school; and way that school staff interact with parents. 

^ Any 5 or more of the following: Told student a story; did arts and crafts with student; played sports, active games or exercised together; worked on projects 
such as building, making, or fixing something with student not as a chore; talked with student about family history or ethnic heritage; and played board games or 
did puzzles with student. 

^ Any 3 or more of the following: Visited a library; visited a bookstore; went to a play, concert, or other live show; visited an art gallery, museum, or historical 
site; visited a zoo or aquarium; attended an event sponsored by a community, religious, or ethnic group; and attended an athletic or sporting event (outside of 
school) in which the child was not a player. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Parent and Family Involvement in Education (PFI) Survey of the National 
Household Education Surveys Program, 2007. 




G-9 



Table G-3. Unadjusted base weight estimates of nonresponse bias for various characteristics from the Adult Education for Work- 
Related Reasons (AEWR) Survey: NHES:2007 Bias Study 





Redueed treatment AEWR 
respondents 


Full treatment AEWR respondents 


Bias' 




Relative bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Partieipates in adult edueation for 
work-related reasons 


433 


40.1 


1.63 


495 


39.4 


1.42 


0.7 


0.57 


1.8 


Partieipates in employer-supported 
AEWR 


307 


69.5 


3.43 


347 


69.4 


3.16 


0.1 


0.49 


0.1 


Partieipates in distanee edueation 


251 


58.9 


2.75 


285 


58.4 


2.28 


0.4 


0.80 


0.9 


Partieipates in program to earn a 
eollege or university degree 


108 


9.6 


0.92 


124 


9.5 


0.83 


0.1 


0.27 


1.1 


Partieipates in program to earn a 
voeational or teehnieal diploma 


48 


4.2 


0.63 


56 


4.3 


0.61 


-0.1 


0.23 


-2.3 


Partieipates in formal 

apprentieeship program 


5! 


0.4! 


0.20! 


5! 


0.3! 


0.18! 


# 


0.02 


# 


Partieipates in work-related 
training or eourses 


368 


34.1 


1.62 


416 


33.3 


1.55 


0.8 


0.42 


2.4 


Partieipates in 4 or more informal 
work-related learning aetivities^ 


290 


29.1 


2.22 


324 


28.3 


1.88 


0.8 


0.65 


2.8 


Has any eondition that limits ability 
to work 


139 


15.1 


2.26 


153 


14.5 


2.15 


0.6 


0.30 


4.1 



See notes at end of table. 
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Table G-3. Unadjusted base weight estimates of nonresponse bias for various characteristics from the Adult Education for Work- 
Related Reasons (AEWR) Survey: NHES:2007 Bias Study — Continued 





Redueed treatment AEWR 
respondents 


Full treatment AEWR respondents 


Bias' 




Relative bias^ 


Charaeteristie 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Sample 

size 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Standard 

error 


Estimate 

(pereent) 


Age 


1 6 to 24 years 


89 


11.6 


1.45 


104 


11.8 


1.51 


-0.2 


0.38 


-1.7 


25 to 34 years 


143 


17.0 


1.60 


168 


17.3 


1.69 


-0.3 


0.37 


-1.7 


35 to 44 years 


111 


15.0 


1.46 


131 


15.2 


1.29 


-0.2 


0.41 


-1.3 


45 to 54 years 


195 


21.2 


1.04 


217 


20.8 


0.87 


0.4 


0.35 


1.9 


55 years and older 


393 


35.1 


1.86 


445 


34.9 


1.62 


0.3 


0.56 


0.6 


Census region 


Northeast 


233 


27.0 


5.61 


259 


26.9 


5.46 


0.1 


0.60 


0.4 


Midwest 


207 


20.4 


3.83 


235 


20.2 


3.67 


0.2 


0.43 


1.0 


South 


260 


28.2 


2.25 


314 


29.3 


2.06 


-1.1 


0.65 


-3.8 


West 


231 


24.4 


1.87 


257 


23.6 


1.78 


0.8 


0.43 


3.4 


Home tenure 


Own 


693 


72.5 


2.52 


794 


73.2 


2.51 


-0.7 


0.53 


-1.0 


Rent/other 


238 


27.5 


2.52 


271 


26.8 


2.51 


0.7 


0.53 


2.6 


Household size 


1 person 


245 


14.8 


1.30 


278 


14.6 


1.08 


0.2 


0.37 


1.4 


More than 1 person 


686 


85.2 


1.30 


787 


85.4 


1.08 


-0.2 


0.37 


-0.2 


Marital status 


Never married 


217 


23.2 


1.67 


249 


23.2 


1.71 


# 


0.52 


# 


Currently married 


468 


56.3 


1.77 


531 


55.8 


1.66 


0.5 


0.65 


0.9 


Other 


246 


20.5 


1.69 


285 


21.0 


1.37 


-0.5 


0.52 


-2.4 



See notes at end of table. 
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Table G-3. Unadjusted base weight estimates of nonresponse bias for various characteristics from the Adult Education for Work- 
Related Reasons (AEWR) Survey: NHES:2007 Bias Study — Continued 



Reduced treatment AEWR 

respondents Full treatment AEWR respondents Bias* Relative bias^ 



Characteristic 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Sample 

size 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Standard 

error 


Estimate 

(percent) 


Flousehold income 


Less than $15,000 


154 


15.3 


1.30 


194 


16.7 


1.18 


-1.4 


0.44 


-8.4 


$15,001 to $30,000 


173 


19.1 


2.27 


212 


20.2 


1.80 


-1.1 


0.68 


-5.4 


$30,001 to $50,000 


171 


19.8 


1.68 


188 


19.0 


1.67 


0.8 


0.31 


4.2 


More than $50,000 


433 


45.8 


1.98 


471 


44.1 


1.49 


1.6 


0.81 


3.9 


Language spoken most at home 


English 


876 


92.1 


1.84 


999 


91.9 


1.75 


0.3 


0.21 


0.2 


Spanish 


33 


5.3 


1.55 


39 


5.2 


1.50 


0.1 


0.25 


1.9 


Other language 


15 


1.9 


0.82 


18 


1.9 


0.76 


-0.1 


0.17 


# 


English and Spanish equally 


7 


0.7 


0.29 


8 


0.7 


0.23 


-0.1 


0.17 


# 


English and other language 


equally 


# 


# 


# 


t 




t 




t 


t 


Employment status 


Employed 


551 


59.4 


2.42 


627 


59.1 


2.16 


0.3 


0.54 


0.5 


Unemployed but looking for 


work 


36 


4.8 


1.38 


44 


5.0 


1.17 


-0.2 


0.39 


-4.0 


Not in the labor force 


344 


35.9 


1.98 


394 


35.9 


1.81 


# 


0.67 


# 



# Rounds to zero. 

! Interpret with caution. 

J Reporting standards not met. 

* Bias is estimated as the difference between the reduced treatment AEWR respondents estimate and the full treatment AEWR respondents estimate. 

^ Relative bias is estimated as the bias estimate divided by the full treatment AEWR respondents estimate, and is expressed as a percentage. 

^Any 4 or more of the following: Received on-the-job demonstrations of equipment, techniques, or procedures by a supervisor or coworker; received other 
supervised training or mentoring on the job; self-paced study using books, procedures manuals, audio tapes, or videos; self-paced study using computer-based 
software tutorials; attended “brown-bag” or informal presentations; attended conferences, trade shows, or conventions related to the adult’s work or profession; 
and read professional journals, trade publications, or work-related magazines. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Adult Education for Work-Related Reasons Survey of the National 
Flousehold Education Surveys Program, 2007. 
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